Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
One possible solution that wallets could adopt when blocks fill up, would be to abandon the p2p network for transaction propagation altogether, and instead work directly with a handful of the largest miners and pools to get transactions into blocks. The miners could auction off space in their blocks with the guarantee that they will be included in the order accepted. We'd lose the peer-to-peer nature of sending transactions for a federated service operator style model, but avoid the problem of unpredictable transaction failure. Also, unlike replace-by-fee, this would allow for a send-and-forget usage pattern, as well as known up-front fees. Something like this is certainly what I imagine the large hosted wallet companies will end up moving to. Aaron On Thursday, June 11, 2015, Mike Hearn wrote: > > Re: "dropped in an unpredictable way" - transactions would be dropped >> > lowest fee/KB first, a completely predictable way. >> >> Quite agreed. > > > No, Aaron is correct. It's unpredictable from the perspective of the user > sending the transaction, and as they are the ones picking the fees, that is > what matters. > -- Aaron Voisine co-founder and CEO breadwallet.com -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Miners: You'll (very likely) need to upgrade your Bitcoin Core node soon to support BIP66
Summary --- The BIP66 soft-fork recently passed the 75% support threshold. This means that 75% of the hashing power has upgraded to support BIP66; 25% of the hashing power has not. Once 95% of the hashing power has upgraded, blocks created by the 5% who have not upgraded will be rejected. If you operate a pool, solo-mine, or mine on p2pool you'll very likely need to upgrade your Bitcoin Core node to support the BIP66 soft-fork, or your blocks will be rejected. If you only sell your hashing power to a centralized pool you do not need to do anything. How does the Bitcoin protocol measure BIP66 support? Miners that have upgraded to support BIP66 create blocks with the version field set to 3; non-upgraded miners set the version to 2. Bitcoin Core measures BIP66 support by counting how many blocks with version >= 3 exist in the blockchain within the last 1000 blocks. If 750 out of the last 1000 blocks support BIP66, blocks with the version set to >= 3 that do not follow the BIP66 rules are rejected; if 950 out of the last 1000 blocks support BIP66, blocks with version < 3 are rejected. When will the 95% threshold be reached? --- It's unknown exactly when the 95% threshold will be reached. The BIP34 soft-fork went from 75% to 95% support in a about two weeks, however more or less time is possible; it's possible that the 95% threshold will be reached in just a few days. How can I monitor BIP66 adoption? - See Pieter Wuille's graphs: http://bitcoin.sipa.be/ver-ever.png http://bitcoin.sipa.be/ver-50k.png http://bitcoin.sipa.be/ver-10k.png http://bitcoin.sipa.be/ver-2k.png The 'ever' and '50k' graphs show the 75% and 95% thresholds. What software supports support BIP66? - Bitcoin Core releases later than v0.10.0 support BIP66. In addition, v0.9.5 supports BIP66, however we recommend that you upgrade to v0.10.2 If you run a pool, you may also need to upgrade your pool software as well. For instance, eloipool versions prior to May 22nd 2015, git commit f5f4ea636fb38f38e6d9a04aad1f04427556a4bc, do not support BIP66. (For Eloipool, cb8a5e8fbb4bfdfe9e35f670082603caff65e1b2 is a clean merge that should work for any branch more recent than 2013 April 6) Solo miners and decentralised miners using GBT need to also update their mining software to a currently supported version of BFGMiner to get support for v3 blocks. The official BFGMiner binaries include this update with 5.1.0, 4.10.2, and 3.10.9. What is BIP66? -- BIP66 - "Strict DER signatures" - is a soft-fork that tightens the rules for signature verification, specifically the way that signatures are encoded. The Bitcoin Core implementation currently relies on OpenSSL for signature validation, which means it implicitly defines Bitcoin's block validity rules. Unfortunately, OpenSSL is not designed for consensus-critical behaviour (it does not guarantee bug-for-bug compatibility between versions), and thus changes to it can - and have - affected Bitcoin software. (see CVE-2014-8275) By tightening these rules BIP66 reduces the risk that changes to OpenSSL will cause forks in the Bitcoin blockchain, as seen previously by the March 2013 fork. Secondly reducing our dependency on OpenSSL is a step towards replacing OpenSSL with libsecp256k1, a signature validation library from Pieter Wuille and Gregory Maxwell, that is designed for consensus-critical applications, as well as being significantly faster than OpenSSL. Is it possible that the BIP66 soft-fork will not happen? In theory yes, though it is unlikely and rejection of BIP66 would be a very ugly process. Unfortunately the existing soft-fork mechanism provides no mechanism for a soft-fork to expire, so once set in motion there is no clean way to stop a soft-fork. There is a proposal from Wuille/Maxwell/Todd, to reform how soft-forks are adopted that aims to fix this issue, as well as allow multiple soft-forks be adopted in parallel: http://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg07863.html -- 'peter'[:-1]@petertodd.org 127ab1d576dc851f374424f1269c4700ccaba2c42d97e778 signature.asc Description: Digital signature -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
> A Header-PoW-verifying client could still be given all transactions in a recent block, from which it can see the in-band fees directly. You don't know the fees paid by any given transaction unless you also have all it's inputs. Transaction inputs do not include an amount. You could however get the average fee-per-kb paid by all transactions in a block by looking at the coinbase transaction, subtracting the block reward, and dividing by the size of block minus the header. Aaron Voisine co-founder and CEO breadwallet.com On Thu, Jun 11, 2015 at 11:30 AM, Nathan Wilcox wrote: > On Wed, Jun 10, 2015 at 2:03 PM, Peter Todd wrote: > >> On Wed, Jun 10, 2015 at 02:00:27PM -0600, Nathan Wilcox wrote: >> > On Wed, Jun 10, 2015 at 1:19 PM, Aaron Voisine >> wrote: >> > >> > > It could be done by agreeing on a data format and encoding it in an >> > > op_return output in the coinbase transaction. If it catches on it >> could >> > > later be enforced with a soft fork. >> > > >> > > >> > Sounds plausible, except SPV protocols would need to include this >> coinbase >> > txn if it's going to help SPV clients. (Until a softfork is activated, >> SPV >> > clients should not rely on this encoding, since until that time the >> results >> > can be fabricated by individual miners.) >> >> Fee stats can always be fabricated by individual miners because fees can >> be paid out-of-band. >> >> > This is a point I hadn't considered carefully before. I don't understand > the marketplace here or why miners would want to move fees outside of > explicit inband fees. Implicit in this proposal is that the statistics only > cover in-band data, because that's the scope of consensus rules, and thus > the proposal is only as useful as the information of in-band fees is useful. > > I've also noticed a detracting technical argument given a particular > tradeoff: > > A Header-PoW-verifying client could still be given all transactions in a > recent block, from which it can see the in-band fees directly. The > trade-off is the size of those transactions versus the need to alter any > consensus rules or do soft forks. > > Notice how this trade-off's costs change with maximum block size. > > > > >> -- >> 'peter'[:-1]@petertodd.org >> 1245bd2f5c99379ee76836227ded9c08324894faabc0d27f >> > > > > -- > Nathan Wilcox > Least Authoritarian > > email: nat...@leastauthority.com > twitter: @least_nathan > PGP: 11169993 / AAAC 5675 E3F7 514C 67ED E9C9 3BFE 5263 1116 9993 > -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
On Wed, Jun 10, 2015 at 2:03 PM, Peter Todd wrote: > On Wed, Jun 10, 2015 at 02:00:27PM -0600, Nathan Wilcox wrote: > > On Wed, Jun 10, 2015 at 1:19 PM, Aaron Voisine > wrote: > > > > > It could be done by agreeing on a data format and encoding it in an > > > op_return output in the coinbase transaction. If it catches on it could > > > later be enforced with a soft fork. > > > > > > > > Sounds plausible, except SPV protocols would need to include this > coinbase > > txn if it's going to help SPV clients. (Until a softfork is activated, > SPV > > clients should not rely on this encoding, since until that time the > results > > can be fabricated by individual miners.) > > Fee stats can always be fabricated by individual miners because fees can > be paid out-of-band. > > This is a point I hadn't considered carefully before. I don't understand the marketplace here or why miners would want to move fees outside of explicit inband fees. Implicit in this proposal is that the statistics only cover in-band data, because that's the scope of consensus rules, and thus the proposal is only as useful as the information of in-band fees is useful. I've also noticed a detracting technical argument given a particular tradeoff: A Header-PoW-verifying client could still be given all transactions in a recent block, from which it can see the in-band fees directly. The trade-off is the size of those transactions versus the need to alter any consensus rules or do soft forks. Notice how this trade-off's costs change with maximum block size. > -- > 'peter'[:-1]@petertodd.org > 1245bd2f5c99379ee76836227ded9c08324894faabc0d27f > -- Nathan Wilcox Least Authoritarian email: nat...@leastauthority.com twitter: @least_nathan PGP: 11169993 / AAAC 5675 E3F7 514C 67ED E9C9 3BFE 5263 1116 9993 -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
On Thu, Jun 11, 2015 at 7:10 AM, Peter Todd wrote: > On Wed, Jun 10, 2015 at 02:18:30PM -0700, Aaron Voisine wrote: > > The other complication is that this will tend to be a lagging indicator > > based on network congestion from the last time you connected. If we > assume > > that transactions are being dropped in an unpredictable way when blocks > are > > full, knowing the network congestion *right now* is critical, and even > then > > you just have to hope that someone who wants that space more than you do > > doesn't show up after you disconnect. > > Hence the need for ways to increase fees on transactions after initial > broadcast like replace-by-fee and child-pays-for-parent. > > I haven't looked closely at replace-by-fee yet, but I assume this is a non-consensus change to mempool mechanics. To me, this seems like the "actuator" side of fee mechanics: it provides a transaction sender a way to influence the system. By contrast, learning about fees is the "sensor" side of fee mechanics. Consider how a replace-by-fee wallet makes fee decisions. When does it replace by fee? It needs feedback in one of two forms: a. Direct feedback from a trnasaction relay service, or: b. Information in the blockchain, which is verified by all verifying nodes and refined by all miners. The first kind of information is quite acceptable and practical for many use cases, but leave the wallet vulnerable to fabrications by that service. This vulnerability is precisely what SPV security intends to mitigate, right? With only information type a, a (non-SPV) wallet can "shop around" to find competing services, and this should work pretty well, provided the wallet can discover those competing services. If, OTOH, the wallet has access to information type b, it now has "perfect competition" across all such services, even when it can't discover the low-priced services directly. This actual-fees-in-the-actual-block-chain information of type b seem like a powerful source of pricing information, and if SPV security is already valuable for other reasons, then it seems natural to leverage that value. Re: "dropped in an unpredictable way" - transactions would be dropped > lowest fee/KB first, a completely predictable way. > > -- > 'peter'[:-1]@petertodd.org > 124bae79afdcee9267b4e6f8137758b8b4135455cd8e3bfd > > > -- > > ___ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > -- Nathan Wilcox Least Authoritarian email: nat...@leastauthority.com twitter: @least_nathan PGP: 11169993 / AAAC 5675 E3F7 514C 67ED E9C9 3BFE 5263 1116 9993 -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
> > > Re: "dropped in an unpredictable way" - transactions would be dropped > > lowest fee/KB first, a completely predictable way. > > Quite agreed. No, Aaron is correct. It's unpredictable from the perspective of the user sending the transaction, and as they are the ones picking the fees, that is what matters. -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
On 6/11/2015 6:10 AM, Peter Todd wrote: > On Wed, Jun 10, 2015 at 02:18:30PM -0700, Aaron Voisine wrote: >> The other complication is that this will tend to be a lagging indicator >> based on network congestion from the last time you connected. If we assume >> that transactions are being dropped in an unpredictable way when blocks are >> full, knowing the network congestion *right now* is critical, and even then >> you just have to hope that someone who wants that space more than you do >> doesn't show up after you disconnect. > Hence the need for ways to increase fees on transactions after initial > broadcast like replace-by-fee and child-pays-for-parent. > > Re: "dropped in an unpredictable way" - transactions would be dropped > lowest fee/KB first, a completely predictable way. Quite agreed. Also, transactions with unconfirmed inputs should be among the first to get dropped, as discussed in the "Dropped-transaction spam" thread. Like all policy rules, either of these works in proportion to its deployment. Be advised that pull request #6068 emphasizes the view that the network will never have consistent mempool/relay policies, and on the contrary needs a framework that supports and encourages pluggable, generally parameterized policies that could (some might say should) conflict wildly with each other. It probably doesn't matter that much. Deploying a new policy still wouldn't be much easier than deploying a patched version. I mean, nobody has proposed a policy rule engine yet (oops). -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
Peter Todd wrote: > Re: "dropped in an unpredictable way" - transactions would be dropped > lowest fee/KB first, a completely predictable way. It would be 'completely predictable' for whoever knew the state and policies of a miner's mempool, but from an end user's perspective that wouldn't matter much: The end users wouldn't know if their transaction(s) would make the cut or not, somewhere in the network, and by what time. They (obviously) won't know what miners will find the next block(s), they won't know the miners' mempool sizes, potential custom eviction policies, etc. I agree that this can be somewhat remedied by FSSRBF/CPFP, though, provided wallets give users a good (semi-automated?) interface for such transaction replacements/chains. -- Martin Lie -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
On Wed, Jun 10, 2015 at 02:18:30PM -0700, Aaron Voisine wrote: > The other complication is that this will tend to be a lagging indicator > based on network congestion from the last time you connected. If we assume > that transactions are being dropped in an unpredictable way when blocks are > full, knowing the network congestion *right now* is critical, and even then > you just have to hope that someone who wants that space more than you do > doesn't show up after you disconnect. Hence the need for ways to increase fees on transactions after initial broadcast like replace-by-fee and child-pays-for-parent. Re: "dropped in an unpredictable way" - transactions would be dropped lowest fee/KB first, a completely predictable way. -- 'peter'[:-1]@petertodd.org 124bae79afdcee9267b4e6f8137758b8b4135455cd8e3bfd signature.asc Description: Digital signature -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism
> > If we assume that transactions are being dropped in an unpredictable way > when blocks are full, knowing the network congestion *right now* is > critical, and even then you just have to hope that someone who wants that > space more than you do doesn't show up after you disconnect. > Yeah, my proposal is not intended to function correctly with full blocks, as Bitcoin cannot work at all in such a state. It assumes that fees only change slowly and that transactions are being cleared normally. -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Is SourceForge still trustworthy enough to host this list?
On Wed, Jun 10, 2015 at 09:36:23PM +0300, s7r wrote: > The mail list is public, so it's not like the data on it is somehow > sensitive. Sourcefoge is fine, it has a nice web UI where you can browse > the message and sort/order them as you want, etc. > > Why would you want to move to a paid solution? And why would you want > users to have to pay per message? This is the worst idea ever from my > point of view. We want to encourage people to join the community, run > full nodes, ask questions, come with solutions, ideas for improvements > and so on. Everyone should read and write and contribute as much as > possible with ideas in debates. You never know who can have bright ideas > in some contexts. > > Bottom line is so far sourceforge handles the mail lists just fine. I > don't see a single advantage another mail list provider / system could > offer, except some headache and extra work for migration. The software > distribution via sourcefoge was cancelled for obvious reasons which I > fully understand and agree to, but it has nothing to do with the mail > lists. We have way more important things to brainstorm about. I completely agree here. I'm not against migration if a much better option comes along, but e.g. paying for another provider sounds like nonsense when sourceforge does this for free (with some minor annoyances - other providers will have their own). Paying per message is far-fetched, something that could work in economic theory with perfectly spherical people in their perfectly efficient market. In practice the likely result would be a mailing list only used for advertisement and promotion, and technical discussion and release announcements would disappear. BTW for people that *don't* like sourceforge's web archive UI there are some other options via gmane: http://dir.gmane.org/gmane.comp.bitcoin.devel Wladimir -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
