Re: [bitcoin-dev] On mempool policy consistency
I read Antoine's original post on this and got the general gist, and here also, it makes sense, but I'd like to ask: is it necessary that (B, C) in the above not *see* A's opt-out "pre-replacement" (inputs: A1, outputs: A, fees: low; call it TX_2)? I get that they cannot replace it Is it actually true that they cannot replace it? If miners and node operators collude and have the incentive to run a patched version of core, is it still technically impossible to replace? the idea that they suffer financial loss from "ignorant" fee bumping is the part that seems weird to me. Even if they waste resources trying to fee-bump, I agree that this does not appear to be a catastrophe.There doesn't seem to be any technical reason why improvements can't be made to allow B and C to have a better view. Cheers, -Yancy On 2022-11-08 10:28, AdamISZ via bitcoin-dev wrote: Hi aj and list, (questions inline) --- Original Message --- On Thursday, October 27th, 2022 at 18:21, Anthony Towns via bitcoin-dev wrote: Is that true? Antoine claims [1 [1]] that opt-in RBF isn't enough to avoid a DoS issue when utxos are jointly funded by untrusting partners, and, aiui, that's the main motivation for addressing this now. [1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html The scenario he describes is: A, B, C create a tx: inputs: A1, B1, C1 [opts in to RBF] fees: normal outputs: [lightning channel, DLC, etc, who knows] they all analyse the tx, and agree it looks great; however just before publishing it, A spams the network with an alternative tx, double spending her input: inputs: A1 [does not opt in to RBF] fees: low outputs: A If A gets the timing right, that's bad for B and C because they've populated their mempool with the 1st transaction, while everyone else sees the 2nd one instead; and neither tx will replace the other. B and C can't know that they should just cancel their transaction, eg: inputs: B1, C1 [opts in to RBF] fees: 50% above normal outputs: [smaller channel, refund, whatever] and might instead waste time trying to fee bump the tx to get it mined, or similar. What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to solve that problem if they have only opt-in RBF available? I read Antoine's original post on this and got the general gist, and here also, it makes sense, but I'd like to ask: is it necessary that (B, C) in the above not *see* A's opt-out "pre-replacement" (inputs: A1, outputs: A, fees: low; call it TX_2)? I get that they cannot replace it, but the idea that they suffer financial loss from "ignorant" fee bumping is the part that seems weird to me. Clearly TX_2 gets gossiped to other mempools; and understood that it does not replace the TX_1 (the 3-input) in B's mempool, say, but why should they not even hear about it? Is it just a matter of engineering, or is there some deeper problem with that. About this general flavour of attack, it's never been a *big* concern in Joinmarket imo (though, we did until recently have a bug that made this happen *by accident*, i.e. people double spending an input out of a negotiated join, albeit it was rare; but it's ofc definitely *possible* to 'grief' like this, given the ordering of events; maker sends signature, maker broadcasts double spend - 95% of the time they will be first). Interactive protocols are yucky and I think there'll always be griefing possibilities; designing around multiple-rounds of negotiation amongst not always-on participants is even more yucky, so just having a 'taker is in charge of network fee; if it's slow or gets double spent out causing time delay then just wait', combined with 'there really isn't any economic incentive for an attacker' (i.e. ignoring griefing) might sound crappy but it's probably just being realistic. Of course, off-chain contracting has more sophisticated considerations than this. Cheers, AdamISZ/waxwing ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev Links: -- [1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
Bob has staked liquidity in a payment channel with Alice who later double spends the same inputs (at a very low feerate) resulting in a stalemate where neither can spend the UTXOs. I just realized I made a mistake. RBF will always mine the higher fee transaction, so in this case, full-rbf would prevent a transaction from being pinned. On 2022-11-08 15:54, yancy via bitcoin-dev wrote: Peter, It sounds like there are two attack vectors; neither of which require full-rbf (correct me if I'm wrong). 1) Bob has staked liquidity in a payment channel with Alice who later double spends the same inputs (at a very low feerate) resulting in a stalemate where neither can spend the UTXOs. The TX that creates the payment channel with Bob will never be mined since the mining pool sees the double spend? 2) Alice spams the network with a double spend wide enough that the double spend makes it into a block before the remainder of the network sees the first spend. In that case of 1), what if Bob required a opt-in rbf? Wouldn't that solve the issue? Bob could just create a replacement transaction with enough fee to get back his UTXO? For 2) it seems to me that neither full-rbf or opt-in rbf resolves this, although it's a probabilistic attack and requires spamming many nodes. Cheers, -Yancy On 2022-11-07 15:32, Peter Todd wrote: On November 3, 2022 5:06:52 PM AST, yancy via bitcoin-dev wrote: AJ/Antoine et al What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to solve that problem if they have only opt-in RBF available? Assuming Alice is a well funded advisory, with enough resources to spam the network so that enough nodes see her malicious transaction first, how does full-rbf solve this vs. opt-in rbf? First of all, to make things clear, remember that the attacks were talking about are aimed at _preventing_ a transaction from getting mined. Alice wants to cheaply broadcast something with low fees that won't get mined soon (if ever), that prevents a protocol from making forward progress. With full-rbf, who saw what transaction first doesn't matter: the higher fee paying transaction will always(*) replace the lower fee one. With opt-in RBF, spamming the network can beat out the alternative. *) So what's the catch? Well, due to limitations in today's mempool implementation, sometimes we can't fully evaluate which tx pays the higher fee. For example, if Alice spams the network with very _large_ numbers transactions spending that input, the current mempool code doesn't even try to figure out if a replacement is better. But those limitations are likely to be fixable. And even right now, without fixing them, Alice still has to use a lot more money to pull off these attacks with full-rbf. So full-rbf definitely improves the situation even if it doesn't solve the problem completely. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
Peter, It sounds like there are two attack vectors; neither of which require full-rbf (correct me if I'm wrong). 1) Bob has staked liquidity in a payment channel with Alice who later double spends the same inputs (at a very low feerate) resulting in a stalemate where neither can spend the UTXOs. The TX that creates the payment channel with Bob will never be mined since the mining pool sees the double spend? 2) Alice spams the network with a double spend wide enough that the double spend makes it into a block before the remainder of the network sees the first spend. In that case of 1), what if Bob required a opt-in rbf? Wouldn't that solve the issue? Bob could just create a replacement transaction with enough fee to get back his UTXO? For 2) it seems to me that neither full-rbf or opt-in rbf resolves this, although it's a probabilistic attack and requires spamming many nodes. Cheers, -Yancy On 2022-11-07 15:32, Peter Todd wrote: On November 3, 2022 5:06:52 PM AST, yancy via bitcoin-dev wrote: AJ/Antoine et al What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to solve that problem if they have only opt-in RBF available? Assuming Alice is a well funded advisory, with enough resources to spam the network so that enough nodes see her malicious transaction first, how does full-rbf solve this vs. opt-in rbf? First of all, to make things clear, remember that the attacks were talking about are aimed at _preventing_ a transaction from getting mined. Alice wants to cheaply broadcast something with low fees that won't get mined soon (if ever), that prevents a protocol from making forward progress. With full-rbf, who saw what transaction first doesn't matter: the higher fee paying transaction will always(*) replace the lower fee one. With opt-in RBF, spamming the network can beat out the alternative. *) So what's the catch? Well, due to limitations in today's mempool implementation, sometimes we can't fully evaluate which tx pays the higher fee. For example, if Alice spams the network with very _large_ numbers transactions spending that input, the current mempool code doesn't even try to figure out if a replacement is better. But those limitations are likely to be fixable. And even right now, without fixing them, Alice still has to use a lot more money to pull off these attacks with full-rbf. So full-rbf definitely improves the situation even if it doesn't solve the problem completely.___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
Hi aj and list, (questions inline) --- Original Message --- On Thursday, October 27th, 2022 at 18:21, Anthony Towns via bitcoin-dev wrote: > > Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid > a DoS issue when utxos are jointly funded by untrusting partners, and, > aiui, that's the main motivation for addressing this now. > > [1] > https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html > > The scenario he describes is: A, B, C create a tx: > > inputs: A1, B1, C1 [opts in to RBF] > fees: normal > outputs: > [lightning channel, DLC, etc, who knows] > > they all analyse the tx, and agree it looks great; however just before > publishing it, A spams the network with an alternative tx, double > spending her input: > > inputs: A1 [does not opt in to RBF] > fees: low > outputs: A > > If A gets the timing right, that's bad for B and C because they've > populated their mempool with the 1st transaction, while everyone else > sees the 2nd one instead; and neither tx will replace the other. B and > C can't know that they should just cancel their transaction, eg: > > inputs: B1, C1 [opts in to RBF] > fees: 50% above normal > outputs: > [smaller channel, refund, whatever] > > and might instead waste time trying to fee bump the tx to get it mined, > or similar. > > What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to > solve that problem if they have only opt-in RBF available? > > I read Antoine's original post on this and got the general gist, and here also, it makes sense, but I'd like to ask: is it necessary that (B, C) in the above not *see* A's opt-out "pre-replacement" (inputs: A1, outputs: A, fees: low; call it TX_2)? I get that they cannot replace it, but the idea that they suffer financial loss from "ignorant" fee bumping is the part that seems weird to me. Clearly TX_2 gets gossiped to other mempools; and understood that it does not replace the TX_1 (the 3-input) in B's mempool, say, but why should they not even hear about it? Is it just a matter of engineering, or is there some deeper problem with that. About this general flavour of attack, it's never been a *big* concern in Joinmarket imo (though, we did until recently have a bug that made this happen *by accident*, i.e. people double spending an input out of a negotiated join, albeit it was rare; but it's ofc definitely *possible* to 'grief' like this, given the ordering of events; maker sends signature, maker broadcasts double spend - 95% of the time they will be first). Interactive protocols are yucky and I think there'll always be griefing possibilities; designing around multiple-rounds of negotiation amongst not always-on participants is even more yucky, so just having a 'taker is in charge of network fee; if it's slow or gets double spent out causing time delay then just wait', combined with 'there really isn't any economic incentive for an attacker' (i.e. ignoring griefing) might sound crappy but it's probably just being realistic. Of course, off-chain contracting has more sophisticated considerations than this. Cheers, AdamISZ/waxwing ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
> > > With full-rbf, who saw what transaction first doesn't matter: the higher > fee paying transaction will always(*) replace the lower fee one. With > opt-in RBF, spamming the network can beat out the alternative. > incentivised predictability is critical when designing low level protocols, like bitcoin. the knock-on effects of deeper, network-wide predictability are likely beneficial in ways that are hard to predict. for example stuff like the "sabu" protocol might even work if full-rbf is the norm. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
On November 3, 2022 5:06:52 PM AST, yancy via bitcoin-dev wrote: > >AJ/Antoine et al > >> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to >> solve that problem if they have only opt-in RBF available? > >Assuming Alice is a well funded advisory, with enough resources to spam the >network so that enough nodes see her malicious transaction first, how does >full-rbf solve this vs. opt-in rbf? First of all, to make things clear, remember that the attacks were talking about are aimed at _preventing_ a transaction from getting mined. Alice wants to cheaply broadcast something with low fees that won't get mined soon (if ever), that prevents a protocol from making forward progress. With full-rbf, who saw what transaction first doesn't matter: the higher fee paying transaction will always(*) replace the lower fee one. With opt-in RBF, spamming the network can beat out the alternative. *) So what's the catch? Well, due to limitations in today's mempool implementation, sometimes we can't fully evaluate which tx pays the higher fee. For example, if Alice spams the network with very _large_ numbers transactions spending that input, the current mempool code doesn't even try to figure out if a replacement is better. But those limitations are likely to be fixable. And even right now, without fixing them, Alice still has to use a lot more money to pull off these attacks with full-rbf. So full-rbf definitely improves the situation even if it doesn't solve the problem completely. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
On Mon, Oct 31, 2022 at 09:02:02AM -0400, Suhas Daftuar via bitcoin-dev wrote: Sending this email for the sake of repeating a point I made on GitHub for the mailing list audience: > AJ, > > Thanks for the thoughtful post. I think your observations about how we view > mempool policy in the Bitcoin Core project, and how that seems to be > changing in the discussions around `-mempoolfullrbf`, are on-point and > provide a helpful baseline for considering future policy changes. > To those who argue for making fullrbf a default policy on the network (or > even just offering a flag for users to enable fullrbf), I pose this > hypothetical: suppose we deploy the v3 transaction policy proposal (which I > hope will happen in the near future). That policy would restrict the ways > that outputs of a v3 transaction can be spent while the transaction is > unconfirmed, including by limiting the number and size of descendants that > such a transaction can have, and limiting the types of unconfirmed > ancestors that can be included. Suppose in a few years someone proposes > that we add a "-disable_v3_transaction_enforcement" flag to our software, > to let users decide to turn off those policy restrictions and treat v3 > transactions the same as v2, for all the same reasons that could be argued > today with fullrbf: miners might earn more revenue if we allowed multiple > descendant v3 transactions; it's illogical for the recipient of a v3 > transaction to believe what is a fundamentally unenforceable promise of a > sender to not issue more high value children that descend from an > unconfirmed transaction; it's inappropriate for Bitcoin Core to dictate > policy on the network and we should honor user choice to turn off that flag > if that’s what users want; if users are relying on v3’s policy restrictions > for security then that is an unstable model and we should assume it will > get broken[5]. Let's frame this question differently: given that there are potential incentives around a hypothetical `-disable_v3_transaction_enforcement` flag, why are we trying to prevent miners and others from experimenting with incentives around `fullrbf`? Yanking the `mempoolfullrbf` flag from Bitcoin Core v24.0 simply puts a temporary roadblock in the face of full-rbf. Without that roadblock, we might find that some miners do in fact choose to enable it. The sooner we find that out, the sooner we can learn about the incentives involved in that decision. Meanwhile, if we insted put up those roadblocks, we'll be designing mechanisms like v3 blind, without the benefit of seeing how incentives play out fully. This experimentation can't happen on testnet: incentives don't work properly when there isn't money at stake. And the proposed reversion pull-reqs don't even leave the option for testnet anyway. So we're left with one choice: release a full-rbf option, and see what happens on mainnet. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
Peter, There's nothing special about a "full-rbf transaction" other than the fact that it's replacing a previously broadcast transaction that didn't signal replacement. Thanks, this is a piece I haven't seen. It sounds like "full-rbf" policy is fundamentally different from BIP125, where in BIP125 a transaction must signal that it can be replaced. If I'm reading what you said correctly, then "full-rbf" policy will allow the replacement of any transaction, whether it's signaled or not.. Since all the machinery to do replacemnt already exists, adding a full-rbf config flag is particularly trivial. It requires just a single line in the mempool code. Agree the flag is trivial. The interplay between mempool policies may not be trivial. Cheers, -Yancy On 2022-10-31 18:51, Peter Todd wrote: On Mon, Oct 31, 2022 at 06:21:08PM +0100, yancy via bitcoin-dev wrote: Protocol Devs, After reading through this email thread and BIP125, I'm curious if non-rbf nodes will relay full-rbf transactions and vice versa. That is to say, if only one non-rbf node exists on the network, however, every other node implements full-rbf, will the transaction still be propagated? IE can we always guarantee a path through the network for either transaction type no matter what the combination of network policies are? 1) There are nodes that signal full-rbf, and preferentially peer to each other, thus ensuring good transaction propagation. The most recent patch to implement this is: https://github.com/bitcoin/bitcoin/pull/25600 There's enough peers running full-rbf that the last time I started up a new node on a fresh IP address, it happened to have a peer relaying full-rbf replacements to it. And of course, if people want full-rbf to work more reliably, it's very easy to just run some nodes with a large number of outgoing peers. Changing the hard-coded 8 outgoing peers to, say, 800, isn't very hard. 2) There's nothing special about a "full-rbf transaction" other than the fact that it's replacing a previously broadcast transaction that didn't signal replacement. There is not consensus over the mempool, so in certain cases non-full-rbf nodes will in fact broadcast replacements when they didn't happen to receive the "first" transaction first. The latter makes testing full-rbf a bit problematic, as if you don't take special measures to ensure good propagation a small % of the time the "replacement" transaction will in fact be the one that gets gets mined. Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they? I think AJ mentioned this earlier, but adding more configuration options always increases code complexity, and with that, there is likely more unforeseen bugs. However, there is a section of network participants that rely on both types of transaction policy, so from my limited view-point, it seems worth accommodating if possible. Since all the machinery to do replacemnt already exists, adding a full-rbf config flag is particularly trivial. It requires just a single line in the mempool code.___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
AJ/Antoine et al What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to solve that problem if they have only opt-in RBF available? Assuming Alice is a well funded advisory, with enough resources to spam the network so that enough nodes see her malicious transaction first, how does full-rbf solve this vs. opt-in rbf? Cheers, -Yancy On 2022-10-27 19:21, Anthony Towns via bitcoin-dev wrote: On Thu, Oct 27, 2022 at 11:56:45AM +0200, John Carvalho via bitcoin-dev wrote: I took the time to read your whole post. Despite a diplomatic tone, I find your takeaways from all your references to remain conveniently biased for protecting the plan of RBF Yes, I am heavily biased against zeroconf: there's no way I'd personally be willing to trust it for my own incoming funds, no matter how much evidence you show me that it's safe in practice. Show me a million transactions where every single one worked fine, and I'm still going to assume that the payment going to me is going to be the one that makes the error rate tick up from 0% to 0.0001%. That's okay; just because I wouldn't do something, doesn't mean other people shouldn't. It does mean I'm not going to be a particularly good advocate for zeroconf though. I mean, I might still be a fine advocate for giving people time to react, making it clear what's going on, finding ways that might make everyone happy, or just digging it to random technical details; but, for me, I'm more interested in a world where chargebacks are impossible, not where we just make the best of what was possible with technology from five or ten years ago. But that's fine: it just means that people, like yourself, who will tolerate the risks of zeroconf, should be involved in the discussion. You show multiple examples where, when I read them, I assume the next thing you will say will be "so we really should stop trying to impose optional features, particularly when they affect existing use cases" but instead you persist. Sure, that's natural: you read a sign saying "you can have any ice cream you want for 5c" and think "Awesome, who wouldn't want cheap chocolate ice cream!!" and see me going for a Golden Gaytime and think "wtf dude". Different strokes. For me, I see the gmaxwell github comment I quoted saying: There is also a matter of driving competent design rather than lazy first thing that works. and think "yeah, okay, maybe we should be working harder to push lightning adoption, rather than letting people stick with wallet UX from 2015" and have altcoins take over >50% of payment volume. Likewise, There is also a very clear pattern we've seen in the past where people take anything the system lets them do as strong evidence that they have a irrevocable right to use the system in that way, and that their only responsibility-- and if their usage harms the system it's the responsibility of the system to not permit it. seems a pretty good match against your claim "I expect the things I do with Bitcoin today to work FOREVER." Better to nip that thinking in the bud; and even if the best time to do that was years ago, the second best time to do it is still now. By contrast, from the same post, I'd guess you're focussing on: Network behavior is one of the few bits of friction driving good technical design rather than "move fast, break things, and force everyone else onto my way of doing thing rather than discussing the design in public". and thinking "yeah, move fast, break things, force everyone else -- that's exactly what's going on here, and shouldn't be". But that's also okay: even when there is common ground to be found, sometimes it requires actual work to get people who start from different views to get there. The problem is that RBF has already been an option for years, and anyone that wants to use it can. Is that true? Antoine claims [1 [1]] that opt-in RBF isn't enough to avoid a DoS issue when utxos are jointly funded by untrusting partners, and, aiui, that's the main motivation for addressing this now. [1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html The scenario he describes is: A, B, C create a tx: inputs: A1, B1, C1 [opts in to RBF] fees: normal outputs: [lightning channel, DLC, etc, who knows] they all analyse the tx, and agree it looks great; however just before publishing it, A spams the network with an alternative tx, double spending her input: inputs: A1 [does not opt in to RBF] fees: low outputs: A If A gets the timing right, that's bad for B and C because they've populated their mempool with the 1st transaction, while everyone else sees the 2nd one instead; and neither tx will replace the other. B and C can't know that they should just cancel their transaction, eg: inputs: B1, C1 [opts in to RBF] fees: 50% above normal outputs: [smaller channel, refund, whatever] and might instead waste time trying to fee bump the tx to get it mined, or similar. What should folks
Re: [bitcoin-dev] On mempool policy consistency
Hi Suhas, >From my understanding, the main crux of the reasoning exposed in your post would be to solidify the transaction-relay paradigm we have been following during the last years, e.g introducing the carve-out rule specifically for lightning commitment transactions, or more recently version=3 transactions. I think this paradigm could be described summarly as "to each use-case belongs a set of transaction-relay policy rules". Some of this set of rules could aim to non-replacement guarantees to the consumers of transactions signaling under this regime (e.g zeroconf). Another set of rules could provide a high-guarantee that a transaction would always get to a miner, no matter what the state of node's mempools on the network (e.g "maximal rbf" for contracting protocols). First, coming out of my mind, we would have to consider isolation between each set of policy rules, ensuring the signaling mechanism cannot be abused by an attacker to create a new pinning vector. E.g, a hypothetical concern could be BIP125 rules interfering with version=3 policy to block the replacement of better ancestor feerate packages. Further, for each set of policy rules arises the question of internal consistency, again an attacker could abuse them to pin transactions propagation. I think an earlier version of version=3 was suffering from this concern of not scoping potential "junk" ancestors. None of those issues are unsolvable, however we should be well-aware of the non-negligeable design complexity encumbered by transaction-relay protocol developers to achieve the correct goal. There is not only a need to ensure careful policy rules security analysis, but further to communicate well their usage to second-layers and wallets developers (a task far from easy with all the confusions contained by current BIP125). Now, in the evaluation process of a set of policy rules soundness are proposed a few reasoning heuristics: namely that it shouldn't interfere sensibly with a anti-DoS mempool acceptance algorithm, or shouldn't interfere with other protocols on the network, or counter the interests of miners or node operators. I hold the belief the latest question could be the one raising the most concerns. Browsing in the history of Bitcoin Core, I think one of the design goals aimed for has always been to level the playing field between miners, e.g BIP152 improving block transfer latency to reduce orphan rate. Following this principle, we might wonder if our transaction-relay network should guarantee some equal access to transaction information to all the miners. In the present case of a non-replacement policy regime, we could see the following situation to arise, on one side miners deploying private transaction-relay communication channels or API to capture higher fees income from non-standard transactions. On the other-side, transaction issuers or consumers bypass the standard transaction-relay policy rules. Bypass could be motivated by either a zeroconf service double-spend, or faster confirmation of a collaborative transaction. E.g, to reuse the example of unconfirmed transaction chaining, where the sender commit to non-replacement by opting out from the RBF flag, this commitment could be reevaluated in the light of changing network mempools congestion, or liquidity preferences (e.g the quick need to open LN routing channels). The sender could leverage such hypothetical private transaction-relay communication channels to revoke its non-replacement commitment. Therefore discrepancies between a set of policy rules design and miners incentives sounds to lead to informational asymmetries, harmful for the long-term decentralization of the mining ecosystem. Of course, miner incomes asymmetries due to edge in transaction flows access might not be weighted as serious today, in a world where transaction fees contribute to most of the block reward, this is far more worrying! Of course, one position could be to estimate that miner centralization is beyond the scope of responsibility of the Bitcoin Core project. Or at least as a result of lightweighted risks. Such discrepancy between a set of policy rules design and miners incentives could also lead to hidden security risks for second-layers. As we see more security assumptions made on policy rules extension, e.g version=3, a lightning channel counterparty could have a competing interest to forge a raw package suiting better incentives, and as such nullify the security advantage expected. This could be seen as a loose concern, however the last time we have seen an actor deliberately providing non-standard transactions to a miner to break second-layers was yesterday [0]!. From observing other cryptocurrencies spaces, such "MEV-style" attacks could be more and more concerning [1]. I don't think we should assume miners to behave as "network gentlemen", in a world where mining can be anonymous, permissionless and censorship-resistant (e.g Stratum V2 giving back template construction to miners
Re: [bitcoin-dev] On mempool policy consistency
> I think that's a huge oversimplification of "rational" -- otherwise you might as well say that deliberately pinning txs is also rational, because it allows the person doing the pinning to steal funds from their counterparty by forcing a timeout to expire. To be clear, a pinner is attempting to *not* pay the most fees, by definition. If we're somehow sure something is a pin, we should not allow it, because miners rationally do not want it vs an "honest" bid for fees. V3 design is one attempt to carve out a safe space for fee bidding. Carving out a safe space for *non-bidding* is not the same thing. I think this mostly boils down having knobs or not. I'm fine with knobs with paternalistic defaults, especially when a non-zero percentage of users disagree with a value in either direction. Greg On Tue, Nov 1, 2022 at 11:07 PM Anthony Towns wrote: > On Mon, Oct 31, 2022 at 12:25:46PM -0400, Greg Sanders via bitcoin-dev > wrote: > > For 0-conf services we have potential thieves who are willing > > to *out-bid themselves* to have funds come back to themselves. It's not a > > "legitimate" use-case, but a rational one. > > I think that's a huge oversimplification of "rational" -- otherwise > you might as well say that deliberately pinning txs is also rational, > because it allows the person doing the pinning to steal funds from their > counterparty by forcing a timeout to expire. > > There's no need for us as developers, or us as node operators, to support > every use case that some individual might find rational at some point in > time. After all, it might be individually rational for someone to want the > subsidy to stop decreasing, or to include 8MB of transactions per block. > > Note that it's also straightforwardly rational and incentive compatible > for miners to not want this patch to be available, under the following > scenario: > > - a significant number of on-chain txs are for zeroconf services > - fee income would be reduced if zeroconf services went away >(both directly due to the absence of zeroconf payments, and by >reducing mempool pressure, reducing fee income from the on-chain txs >that remain) > - miners adopting fullrbf would cause zeroconf services to go away, >(and won't enable a comparable volume of new services that generates >comparable transaction volume) > - including the option in core would make other miners adopting >fullrbf more likely > > I think the first three of those are fairly straightforward and objective, > at least at this point in time. The last is just a risk; but without > any counterbalancing benefit, why take it? > > Gaining a few thousand sats due to high feerate replacement txs from > people exploiting zeroconf services for a few months before all those > services shutdown doesn't make up for the lost fee income over the months > or years it might have otherwise taken people to naturally switch to > some better alternative. > > Even if fullrbf worked for preventing pinning that likely doesn't directly > result in much additional fee income: once you know that pinning doesn't > work, you just don't try it, which means there's no opportunity for > miners to profit from a bidding war from the pinners counterparties > repeatedly RBFing their preferred tx to get it mined. > > That also excludes second order risks: if you can't do zeroconf with BTC > anymore, do you switch to ERC20 tokens, and then trade your BTC savings > for ETH or USDT, and do enough people do that to lower the price of BTC? > If investors see BTC being less used for payments, does that lower their > confidence in bitcoin's future, and cause them to sell? > > > Removing a > > quite-likely-incentive-compatible option from the software just > encourages > > miners to adopt an additional patch > > Why shouldn't miners adopt an additional patch if they want some unusual > functionality? > > Don't we want/expect miners to have the ability to change the code in > meaningful ways, at a minimum to be able to cope with the scenario where > core somehow gets coopted and releases bad code, or to be able to deal > with the case where an emergency patch is needed? > > Is there any evidence miners even want this option? Peter suggested > that some non-signalling replacements were being mined already [0], but > as far as I can see [1] all of those are simply due to the transaction > they replaced not having propagated in the first place (or having been > evicted somehow? hard to tell without any data on the original tx). > > [0] > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021012.html > [1] https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292692367 > > > 2) Forcing miners to honor fees left on the table with respect to 0-conf, > > or forcing them to run a custom patchset to go around it, is a step > > backwards. > > As you already acknowledged, any miner that wants this behaviour can just > pick up the patch (or could run Knots, which already has
Re: [bitcoin-dev] On mempool policy consistency
On Mon, Oct 31, 2022 at 12:25:46PM -0400, Greg Sanders via bitcoin-dev wrote: > For 0-conf services we have potential thieves who are willing > to *out-bid themselves* to have funds come back to themselves. It's not a > "legitimate" use-case, but a rational one. I think that's a huge oversimplification of "rational" -- otherwise you might as well say that deliberately pinning txs is also rational, because it allows the person doing the pinning to steal funds from their counterparty by forcing a timeout to expire. There's no need for us as developers, or us as node operators, to support every use case that some individual might find rational at some point in time. After all, it might be individually rational for someone to want the subsidy to stop decreasing, or to include 8MB of transactions per block. Note that it's also straightforwardly rational and incentive compatible for miners to not want this patch to be available, under the following scenario: - a significant number of on-chain txs are for zeroconf services - fee income would be reduced if zeroconf services went away (both directly due to the absence of zeroconf payments, and by reducing mempool pressure, reducing fee income from the on-chain txs that remain) - miners adopting fullrbf would cause zeroconf services to go away, (and won't enable a comparable volume of new services that generates comparable transaction volume) - including the option in core would make other miners adopting fullrbf more likely I think the first three of those are fairly straightforward and objective, at least at this point in time. The last is just a risk; but without any counterbalancing benefit, why take it? Gaining a few thousand sats due to high feerate replacement txs from people exploiting zeroconf services for a few months before all those services shutdown doesn't make up for the lost fee income over the months or years it might have otherwise taken people to naturally switch to some better alternative. Even if fullrbf worked for preventing pinning that likely doesn't directly result in much additional fee income: once you know that pinning doesn't work, you just don't try it, which means there's no opportunity for miners to profit from a bidding war from the pinners counterparties repeatedly RBFing their preferred tx to get it mined. That also excludes second order risks: if you can't do zeroconf with BTC anymore, do you switch to ERC20 tokens, and then trade your BTC savings for ETH or USDT, and do enough people do that to lower the price of BTC? If investors see BTC being less used for payments, does that lower their confidence in bitcoin's future, and cause them to sell? > Removing a > quite-likely-incentive-compatible option from the software just encourages > miners to adopt an additional patch Why shouldn't miners adopt an additional patch if they want some unusual functionality? Don't we want/expect miners to have the ability to change the code in meaningful ways, at a minimum to be able to cope with the scenario where core somehow gets coopted and releases bad code, or to be able to deal with the case where an emergency patch is needed? Is there any evidence miners even want this option? Peter suggested that some non-signalling replacements were being mined already [0], but as far as I can see [1] all of those are simply due to the transaction they replaced not having propagated in the first place (or having been evicted somehow? hard to tell without any data on the original tx). [0] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021012.html [1] https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292692367 > 2) Forcing miners to honor fees left on the table with respect to 0-conf, > or forcing them to run a custom patchset to go around it, is a step > backwards. As you already acknowledged, any miner that wants this behaviour can just pick up the patch (or could run Knots, which already has the feature enabled by default). It's simply false to say miners are being forced to do anything, no matter what we do here. If the direction you're facing is one where you're moving towards making life easier for people to commit fraud, and driving away businesses that aren't doing anyone harm, without achieving anything much else; then taking a step backwards seems like a sensible thing to do to me. (I remain optimistic about coming up with better RBF policy, and willing to be gung ho about everyone switching over to it even if it does kill off zeroconf, provided it actually does some good and we give people 6 months or more notice that it's definitely happening and what exactly the new rules will be, though) Cheers, aj ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
On Mon, Oct 31, 2022 at 06:21:08PM +0100, yancy via bitcoin-dev wrote: > > Protocol Devs, > > After reading through this email thread and BIP125, I'm curious if non-rbf > nodes will relay full-rbf transactions and vice versa. That is to say, if > only one non-rbf node exists on the network, however, every other node > implements full-rbf, will the transaction still be propagated? IE can we > always guarantee a path through the network for either transaction type no > matter what the combination of network policies are? 1) There are nodes that signal full-rbf, and preferentially peer to each other, thus ensuring good transaction propagation. The most recent patch to implement this is: https://github.com/bitcoin/bitcoin/pull/25600 There's enough peers running full-rbf that the last time I started up a new node on a fresh IP address, it happened to have a peer relaying full-rbf replacements to it. And of course, if people want full-rbf to work more reliably, it's very easy to just run some nodes with a large number of outgoing peers. Changing the hard-coded 8 outgoing peers to, say, 800, isn't very hard. 2) There's nothing special about a "full-rbf transaction" other than the fact that it's replacing a previously broadcast transaction that didn't signal replacement. There is not consensus over the mempool, so in certain cases non-full-rbf nodes will in fact broadcast replacements when they didn't happen to receive the "first" transaction first. The latter makes testing full-rbf a bit problematic, as if you don't take special measures to ensure good propagation a small % of the time the "replacement" transaction will in fact be the one that gets gets mined. > > Does fullrbf offer any benefits other than breaking zeroconf > > business practices? If so, what are they? > > I think AJ mentioned this earlier, but adding more configuration options > always increases code complexity, and with that, there is likely more > unforeseen bugs. However, there is a section of network participants that > rely on both types of transaction policy, so from my limited view-point, it > seems worth accommodating if possible. Since all the machinery to do replacemnt already exists, adding a full-rbf config flag is particularly trivial. It requires just a single line in the mempool code. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
Protocol Devs, After reading through this email thread and BIP125, I'm curious if non-rbf nodes will relay full-rbf transactions and vice versa. That is to say, if only one non-rbf node exists on the network, however, every other node implements full-rbf, will the transaction still be propagated? IE can we always guarantee a path through the network for either transaction type no matter what the combination of network policies are? Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they? I think AJ mentioned this earlier, but adding more configuration options always increases code complexity, and with that, there is likely more unforeseen bugs. However, there is a section of network participants that rely on both types of transaction policy, so from my limited view-point, it seems worth accommodating if possible. Cheers, -Yancy On 2022-10-31 17:25, Greg Sanders via bitcoin-dev wrote: Thanks for your full thoughts Suhas, The idea of V3 is that we're currently leaving fees on the table by allowing use-cases to be pinned, not that we like Lightning and we think miners should stop being profit maximizing somehow to enable safer/better layer 2 systems. If someone wants to bump fees for V3 transactions(or replace them!), there's a much simpler "API" to do so than in legacy policy land. The fact that it disallows more idiotic ways to add more total fees means wallets "shouldn't do that". If it ends up that V3 is disallowing too many "natural" ways to fee bump, that's a strike against the V3 idea and should be discussed. For 0-conf services we have potential thieves who are willing to *out-bid themselves* to have funds come back to themselves. It's not a "legitimate" use-case, but a rational one. I have mostly come around to not pushing for fullrbf due to the issues you mentioned, except taking away the option. Removing a quite-likely-incentive-compatible option from the software just encourages miners to adopt an additional patch if they ever deem it necessary to increase their revenue, even if that revenue is from hurting 0-conf businesses. Maybe putting/leaving in a default-false flag for disabling these "carve outs" is the least bad option. V3 usage patterns shouldn't crumble if a large majority of miners opt out, but 0-conf use cases crumble after a small percentage of adoption. To recap my thoughts: 1) I have put away my fullrbf hats, I will not advocate anyone running it as I think it doesn't really do anything useful for users who aren't trying to double-spend merchants. 2) Forcing miners to honor fees left on the table with respect to 0-conf, or forcing them to run a custom patchset to go around it, is a step backwards. Greg On Mon, Oct 31, 2022 at 11:03 AM Suhas Daftuar via bitcoin-dev wrote: AJ, Thanks for the thoughtful post. I think your observations about how we view mempool policy in the Bitcoin Core project, and how that seems to be changing in the discussions around `-mempoolfullrbf`, are on-point and provide a helpful baseline for considering future policy changes. For a long time I viewed fullrbf as an eventuality and I considered myself to be philosophically supportive of the idea. However, after giving this issue some thought in the past few weeks, I am reversing my thinking on this. Concretely, I will argue that we should continue to maintain a relay policy where replacements are rejected for transactions that don't opt-in to RBF (as described in BIP 125), and moreover, that we should remove the `-mempoolfullrbf` flag from Bitcoin Core's latest release candidate and not plan to release software with that flag, unless (or until) circumstances change on the network, which I'll discuss below. This is, of course, a nuanced topic, and among the considerations is a philosophy of how to think about the relay policy and configuration options that we make available in Bitcoin Core (a consideration that is perhaps unique to that project, but I think relevant for this mailing list). I'll start with some technical issues regarding the benefits of enabling fullrbf on the network. In the current BIP 125 regime, every time a transaction is created, a choice is made whether to subject the transaction to BIP 125's RBF rules or not (based on the sequence values of the inputs). So given that users can already opt-in to RBF, the benefit of a "fullrbf" network policy would be if, somehow, RBF users were still denied the benefits of RBF due to the existence of other transactions that don't opt-in. Along those lines, Antoine Riard brought up[1] a DoS vector that is available to someone who wants to interfere with multi-party funded transactions, and suggested that fullrbf would eliminate the problem. After exploring that question again in this thread (thanks to Greg Sanders for clarifying this to me), I understand that the issue is around ensuring that a multiparty (coinjoin-type) protocol is able to make eventual
Re: [bitcoin-dev] On mempool policy consistency
Thanks for your full thoughts Suhas, The idea of V3 is that we're currently leaving fees on the table by allowing use-cases to be pinned, not that we like Lightning and we think miners should stop being profit maximizing somehow to enable safer/better layer 2 systems. If someone wants to bump fees for V3 transactions(or replace them!), there's a much simpler "API" to do so than in legacy policy land. The fact that it disallows more idiotic ways to add more total fees means wallets "shouldn't do that". If it ends up that V3 is disallowing too many "natural" ways to fee bump, that's a strike against the V3 idea and should be discussed. For 0-conf services we have potential thieves who are willing to *out-bid themselves* to have funds come back to themselves. It's not a "legitimate" use-case, but a rational one. I have mostly come around to not pushing for fullrbf due to the issues you mentioned, except taking away the option. Removing a quite-likely-incentive-compatible option from the software just encourages miners to adopt an additional patch if they ever deem it necessary to increase their revenue, even if that revenue is from hurting 0-conf businesses. Maybe putting/leaving in a default-false flag for disabling these "carve outs" is the least bad option. V3 usage patterns shouldn't crumble if a large majority of miners opt out, but 0-conf use cases crumble after a small percentage of adoption. To recap my thoughts: 1) I have put away my fullrbf hats, I will not advocate anyone running it as I think it doesn't really do anything useful for users who aren't trying to double-spend merchants. 2) Forcing miners to honor fees left on the table with respect to 0-conf, or forcing them to run a custom patchset to go around it, is a step backwards. Greg On Mon, Oct 31, 2022 at 11:03 AM Suhas Daftuar via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > AJ, > > Thanks for the thoughtful post. I think your observations about how we > view mempool policy in the Bitcoin Core project, and how that seems to be > changing in the discussions around `-mempoolfullrbf`, are on-point and > provide a helpful baseline for considering future policy changes. > > For a long time I viewed fullrbf as an eventuality and I considered myself > to be philosophically supportive of the idea. However, after giving this > issue some thought in the past few weeks, I am reversing my thinking on > this. Concretely, I will argue that we should continue to maintain a relay > policy where replacements are rejected for transactions that don't opt-in > to RBF (as described in BIP 125), and moreover, that we should remove the > `-mempoolfullrbf` flag from Bitcoin Core’s latest release candidate and not > plan to release software with that flag, unless (or until) circumstances > change on the network, which I'll discuss below. > > This is, of course, a nuanced topic, and among the considerations is a > philosophy of how to think about the relay policy and configuration options > that we make available in Bitcoin Core (a consideration that is perhaps > unique to that project, but I think relevant for this mailing list). > > I'll start with some technical issues regarding the benefits of enabling > fullrbf on the network. In the current BIP 125 regime, every time a > transaction is created, a choice is made whether to subject the transaction > to BIP 125’s RBF rules or not (based on the sequence values of the > inputs). So given that users can already opt-in to RBF, the benefit of a > “fullrbf” network policy would be if, somehow, RBF users were still denied > the benefits of RBF due to the existence of other transactions that don’t > opt-in. > > Along those lines, Antoine Riard brought up[1] a DoS vector that is > available to someone who wants to interfere with multi-party funded > transactions, and suggested that fullrbf would eliminate the problem. > After exploring that question again in this thread (thanks to Greg Sanders > for clarifying this to me), I understand that the issue is around ensuring > that a multiparty (coinjoin-type) protocol is able to make eventual > progress, by having a candidate multiparty transaction either eventually > confirm or become conflicted with something that has been confirmed, in > which case the double-spend information could be used to start a new > coinjoin round with fewer participants. The concern Antoine and Greg have > brought up is that non-rbf transactions can persist in the mempool > ~indefinitely (at a low feerate and not subject to replacement) and > interfere with progress being made in a coinjoin protocol. > > However, it seems to me that similar problems exist for such a protocol > even in a fullrbf world, as we understand that term today. I mentioned the > ability for rbf “pinning” to interfere with relay of the multiparty > transaction (even if the conflicting transaction signals for RBF – a set of > large but low feerate conflicting transactions can persist in the
Re: [bitcoin-dev] On mempool policy consistency
AJ, Thanks for the thoughtful post. I think your observations about how we view mempool policy in the Bitcoin Core project, and how that seems to be changing in the discussions around `-mempoolfullrbf`, are on-point and provide a helpful baseline for considering future policy changes. For a long time I viewed fullrbf as an eventuality and I considered myself to be philosophically supportive of the idea. However, after giving this issue some thought in the past few weeks, I am reversing my thinking on this. Concretely, I will argue that we should continue to maintain a relay policy where replacements are rejected for transactions that don't opt-in to RBF (as described in BIP 125), and moreover, that we should remove the `-mempoolfullrbf` flag from Bitcoin Core’s latest release candidate and not plan to release software with that flag, unless (or until) circumstances change on the network, which I'll discuss below. This is, of course, a nuanced topic, and among the considerations is a philosophy of how to think about the relay policy and configuration options that we make available in Bitcoin Core (a consideration that is perhaps unique to that project, but I think relevant for this mailing list). I'll start with some technical issues regarding the benefits of enabling fullrbf on the network. In the current BIP 125 regime, every time a transaction is created, a choice is made whether to subject the transaction to BIP 125’s RBF rules or not (based on the sequence values of the inputs). So given that users can already opt-in to RBF, the benefit of a “fullrbf” network policy would be if, somehow, RBF users were still denied the benefits of RBF due to the existence of other transactions that don’t opt-in. Along those lines, Antoine Riard brought up[1] a DoS vector that is available to someone who wants to interfere with multi-party funded transactions, and suggested that fullrbf would eliminate the problem. After exploring that question again in this thread (thanks to Greg Sanders for clarifying this to me), I understand that the issue is around ensuring that a multiparty (coinjoin-type) protocol is able to make eventual progress, by having a candidate multiparty transaction either eventually confirm or become conflicted with something that has been confirmed, in which case the double-spend information could be used to start a new coinjoin round with fewer participants. The concern Antoine and Greg have brought up is that non-rbf transactions can persist in the mempool ~indefinitely (at a low feerate and not subject to replacement) and interfere with progress being made in a coinjoin protocol. However, it seems to me that similar problems exist for such a protocol even in a fullrbf world, as we understand that term today. I mentioned the ability for rbf “pinning” to interfere with relay of the multiparty transaction (even if the conflicting transaction signals for RBF – a set of large but low feerate conflicting transactions can persist in the mempool and make it difficult for the coinjoin transaction from confirming, at least without attaching a very large fee); and as Greg mentioned in a followup, the BIP 125 rule to only permit 100 transactions to be removed from the mempool at a time during a replacement can also be used to pin a coinjoin protocol in the same way as a non-rbf transaction today. It seems to me that what these multiparty protocols actually need is some sort of "maximal rbf" network policy: a way to guarantee that a transaction which should be desirable for a miner to mine would always get to a miner and considered for inclusion in a block, no matter what the state of node’s mempools on the network. While that sounds like a reasonable thing to want on its face (and worth working on), it's not how opt-in RBF works today, nor is it how transaction relay has ever conceptually worked. We have not, thus far, been able to come up with a total ordering on transaction desirability. Moreover, due to all the DoS issues that exist with transaction relay, there are plenty of seemingly legitimate ways to construct transactions that would not relay well on the network. Relay has only ever been a best-efforts concept, where we carve out a small subset of the entire transaction universe for which we try to optimize propagation. The idea behind this approach is that if every use case we can come up with has some way to achieve its goals using transactions that should (eventually) be able to relay, then users wouldn’t have much demand for transactions that would deviate from the supported policies, and we therefore shouldn’t need to worry too much about incentive compatibility concerns when it comes to transaction types that wouldn’t relay at all, even if they are high feerate. (And when those situations arise where the standard transactions do not accommodate some needed use case, developers typically work to define a policy that is compatible with our anti-DoS goals to support such use
Re: [bitcoin-dev] On mempool policy consistency
Whether that's terrible or not depends on how easy it is to retry (and how likely the retry is to succeed) after a failure -- if a TCP packet fails, it just gets automatically resent, and if that succeeds, there's a little lag, but your connection is still usable I'm not sure if that analogy fits here. A TCP packet will be retried (as opposed to UDP), although usually the retry strategy is because of an underlying connection issue, not a protocol mismatch or in this case "policy" mismatch, right? If network propagation and node discovery becomes an issue, and as Antoine mentions, there becomes a need for competing services as I understand it, could that give rise to indexes and trackers who spider the network to create world view? Perhaps it's a bad idea since "third party" trackers are security holes, however, to my knowledge, we already have "trusted" nodes during the initial bootstrap process. Even so, I don't think we could preclude such solutions from occurring organically if the need is arises. Cheers, -Yancy On 2022-10-30 02:02, Anthony Towns via bitcoin-dev wrote: On Fri, Oct 28, 2022 at 09:45:09PM -1000, David A. Harding via bitcoin-dev wrote: I think this might be understating the problem. A 95% chance of having an outbound peer accept your tx conversely implies 1 in 20 payments will fail to propagate on their initial broadcast. Whether that's terrible or not depends on how easy it is to retry (and how likely the retry is to succeed) after a failure -- if a TCP packet fails, it just gets automatically resent, and if that succeeds, there's a little lag, but your connection is still usable. I think it's *conceivable* that a 5% failure rate could be detectable and automatically rectified. Not that I have a good idea how you'd actually do that, in a way that's efficient/private/decentralised... Some napkin math: there are about 250,000 transactions a day; if we round that up to 100 million a year and assume we only want one transaction per year to fail to initially propagate on a network where 30% of nodes have adopted a more permissive policy, lightweight clients will need to connect to over 50 randomly selected nodes.[1] A target failure probability of 1-in-1e8 means: * with 8 connections, you need 90% of the network to support your txs * with 12 connections, you need ~79% * with 24 connections (eg everyone running a long-lived node is listening, so long lived nodes make 12 outbound and receive about ~12 inbound; shortlived nodes just do 24 outbound), you need ~54% So with that success target, and no preferential peering, you need a majority of listening nodes to support your tx's features in most reasonable scenarios, I think. For a more permissive policy only adopted by 10% of nodes, the lightweight client needs to connect to almost 150 nodes. I get 175 connections needed for that scenario; or 153 with a target failure rate of 1-in-10-million. Cheers, aj ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
On Sun, Oct 30, 2022 at 11:02:43AM +1000, Anthony Towns via bitcoin-dev wrote: > > Some napkin math: there are about 250,000 transactions a day; if > > we round that up to 100 million a year and assume we only want one > > transaction per year to fail to initially propagate on a network where > > 30% of nodes have adopted a more permissive policy, lightweight clients > > will need to connect to over 50 randomly selected nodes.[1] > A target failure probability of 1-in-1e8 means: Oh, based on the "receive version message" log entries of a node that only does outbound connections, over the last ~3 weeks I see about 3000 outbound connections (mostly feelers/block-relay-only ones), of which a bunch identify as non-taproot supporting: 10 /Satoshi:0.16.0/: 13 /Satoshi:0.17.0/: 13 /Satoshi:0.17.0.1/: 28 /Satoshi:0.16.3/: 29 /Satoshi:0.19.0.1/: 36 /Satoshi:0.18.1/: 37 /Satoshi:0.19.1/: 39 /Satoshi:0.17.1/: 50 /Satoshi:0.20.0/: 94 /Satoshi:0.21.0/: 95 /Satoshi:0.18.0/: 244 /Satoshi:0.20.1/: Those add up to 688+ of 3065 total; if that's representative, it presumably means a random node connecting to 8 random listening peers has a 6.44-in-1-million chance of only connecting to peers that don't support taproot, ie failing your suggested threshold by a factor of about 644. Cheers, aj ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
On Thu, Oct 27, 2022 at 09:29:47PM +0100, Antoine Riard via bitcoin-dev wrote:
> Let's take the contra.
(I don't think I know that phrase? Is it like "play devil's advocate"?)
> I would say the current post describes the state of Bitcoin Core and
> beyond policy
> rules with a high-degree of exhaustivity and completeness, though itt what
> is, mostly a description. While I think it ends with a set of
> recommendations
It was only intended as a description, not a recommendation for anything.
At this point, the only thing I think I could honestly recommend
that doesn't seem like it comes with massive downsides, is for core to
recommend and implement a particular mempool policy, and only have options
that either make it feasible to scale that policy to different hardware
limitations, and provide options that users can activate en-masse if it
turns out people are doing crazy things in the mempool (eg, a new policy
turns out to be ill-conceived, and it's better to revert to a previous
policy; or a potential spam vector gets exploited at scale).
> What should be actually the design goals and
> principles of Core's transaction-relay propagation rules
> of which mempool accepts ones is a subset?
I think the goals of mempool/relay policy are _really_ simple; namely:
* relay transactions from users to all potential miners, so that
non-mining nodes don't have to directly contact miners to announce
their tx, both for efficiency (your tx can appear in the next block
anyone mines, rather than just the next block you mine) and privacy
(so that miners don't know who a transaction belongs to, so that
users don't have to know who miners are, and so there's no/minimal
correlation between who proposed a tx and who mined the block it
appears in)
* having most of the data that makes up the next block pre-validated
and pre-distributed throughout the network, so that block validation
and relay is much more efficient
> By such design goals, I'm
> thinking either, a qualitative framework, like attacks game for a concrete
> application ("Can we prevent pinning against multi-party Coinjoin ?").
I don't think that even makes sense as a question at that level: you can
only ask questions like that if you already have known mempool policies
across the majority of nodes and miners. If you don't, you have to allow
for the possibility that 99% of hashrate is receiving private blacklists
from OFAC and that one of your coinjoin counterparties is on that list,
eg, and at that point, I don't think pinning is even conceivably solvable.
> I believe we would come up with a
> second-order observation. That we might not be able to satisfy every
> use-case with the standard set of policy rules. E.g, a contracting protocol
> could look for package size beyond the upper bound anti-Dos limit.
One reason that limit is in place is that it the larger the tx is
compared to the block limit, the more likely you are to hit corner cases
where greedily filling a block with the highest fee ratex txs first
is significantly suboptimal. That might mean, eg, that there's 410kvB
of higher fee rate txs than your 600kvB large package, and that your
stuff gets delayed, because the miner isn't clever enough to realise
dropping the 10kvB is worthwhile. Or it might mean that your tx gets
delayed because the complicated analysis takes a minute to run and a
block was mined using the simpler algorithm first. Or it might mean that
some mining startup with clever proprietary software that can calculate
this stuff quickly make substantially more profit than everyone else,
so they start dominating block generation, despite the fact that they're
censoring transactions due to OFAC rules.
> Or even the
> global ressources offered by the network of full-nodes are not high enough
> to handle some application event.
Blocks are limited on average to 4MB-per-10-minutes (6.7kB/second),
and applications certainly shouldn't be being designed to only work if
they can monopolise the entirety of the next few blocks. I don't think
it makes any sense to imagine application events in Bitcoin that exceed
the capacity of a random full node. And equally, even if you're talking
about some other blockchain with higher capacity; I don't really think
it makes sense to call it a "full" node if it can't actually cope with
the demands placed on it by any application that works on that network.
> E.g a Lightning Service Provider doing a
> liquidity maintenance round of all its counterparties, and as such
> force-closing and broadcasting more transactions than can be handled at the
> transaction-relay layer due to default MAX_PEER_TX_ANNOUNCEMENTS value.
MAX_PEER_TX_ANNOUNCEMENTS is 5000 txs, and it's per-peer. If you're an
LSP that's doing that much work, it seems likely that you'd at least
be running a long-lived listening node, so likely have 100+ peers, and
could conceivably simultaneously announce 500k txs distributed across
them, which at 130vB each
Re: [bitcoin-dev] On mempool policy consistency
On Fri, Oct 28, 2022 at 09:45:09PM -1000, David A. Harding via bitcoin-dev wrote: > I think this might be understating the problem. A 95% chance of having > an outbound peer accept your tx conversely implies 1 in 20 payments will > fail to propagate on their initial broadcast. Whether that's terrible or not depends on how easy it is to retry (and how likely the retry is to succeed) after a failure -- if a TCP packet fails, it just gets automatically resent, and if that succeeds, there's a little lag, but your connection is still usable. I think it's *conceivable* that a 5% failure rate could be detectable and automatically rectified. Not that I have a good idea how you'd actually do that, in a way that's efficient/private/decentralised... > Some napkin math: there are about 250,000 transactions a day; if > we round that up to 100 million a year and assume we only want one > transaction per year to fail to initially propagate on a network where > 30% of nodes have adopted a more permissive policy, lightweight clients > will need to connect to over 50 randomly selected nodes.[1] A target failure probability of 1-in-1e8 means: * with 8 connections, you need 90% of the network to support your txs * with 12 connections, you need ~79% * with 24 connections (eg everyone running a long-lived node is listening, so long lived nodes make 12 outbound and receive about ~12 inbound; shortlived nodes just do 24 outbound), you need ~54% So with that success target, and no preferential peering, you need a majority of listening nodes to support your tx's features in most reasonable scenarios, I think. > For a more > permissive policy only adopted by 10% of nodes, the lightweight client > needs to connect to almost 150 nodes. I get 175 connections needed for that scenario; or 153 with a target failure rate of 1-in-10-million. Cheers, aj ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
On 2022-10-26 13:52, Anthony Towns via bitcoin-dev wrote: The cutoff for that is probably something like "do 30% of listening nodes have a compatible policy"? If they do, then you'll have about a 95% chance of having at least one of your outbound peers accept your tx, just by random chance. I think this might be understating the problem. A 95% chance of having an outbound peer accept your tx conversely implies 1 in 20 payments will fail to propagate on their initial broadcast. That seems to me like an unacceptably high failure rate both for the UX of regular payments and for the safety of time-sensitive transactions like onchain HTLC resolutions. Additionally, the less reliable propagation is, the more reliably spy nodes can assume the first IP address they received a transaction from is the creator of that transaction. I think those two problems combine in an especially unfortunate way for lightweight clients. Lightweight clients wanting to find a peer who supports a more permissive policy than most of the network and whose client authors want to provide a good UX (or safety in the case of time sensitive contract protocols like LN) will need to open large numbers of connections, increasing their chance of connecting to a spy node which will associate their IP address with their transaction, especially since lightweight clients can't pretend to be relaying transactions for other users. Some napkin math: there are about 250,000 transactions a day; if we round that up to 100 million a year and assume we only want one transaction per year to fail to initially propagate on a network where 30% of nodes have adopted a more permissive policy, lightweight clients will need to connect to over 50 randomly selected nodes.[1] For a more permissive policy only adopted by 10% of nodes, the lightweight client needs to connect to almost 150 nodes. This also implies that nodes adopting a more restrictive policy degrades UX, safety, and privacy for users of transactions violating that policy. For example, if 30% of nodes used Knots's -spkreuse configuration option and about 50% of transactions reuse scriptPubKeys, then about 9 transactions a day wouldn't initially propagate (assuming 8 randomly selected peers[2]) and lightweight clients who wanted 1-in-100-million safety would need to connect to about 15 random nodes. Towns's post to which I'm replying describes several alternative approaches which mitigate the above problems, but he also documents that they're not without tradeoffs. -Dave [1] (1-0.3)**50 * 100_000_000 =~ 1.8 [2] That assumes every transaction is sent to a different randomly-selected set of peers, which isn't really the case. However, one day $GIANT_EXCHANGE could suddenly be unable to broadcast hundreds or thousands of withdrawal transactions because all of its peers implement a restrictive policy. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
Hi AJ,
Let's take the contra.
I would say the current post describes the state of Bitcoin Core and
beyond policy
rules with a high-degree of exhaustivity and completeness, though itt what
is, mostly a description. While I think it ends with a set of
recommendations on what could be the relations between miners, devs and
node operators w.r.t adopting policy rules, to me it doesn't explore enough
the hard part of the subject. What should be actually the design goals and
principles of Core's transaction-relay propagation rules
of which mempool accepts ones is a subset ? By such design goals, I'm
thinking either, a qualitative framework, like attacks game for a concrete
application ("Can
we prevent pinning against multi-party Coinjoin ?"). Or a quantitative
approach, e.g how fast should be transaction-relay throughput for a given
topology, or how much accurate is the block template against usual blocks
[0]
Assuming we would have technical principles and goals guiding our
development process, of which the discussion and revision should be an
inherent part of the process itself, I believe we would come up with a
second-order observation. That we might not be able to satisfy every
use-case with the standard set of policy rules. E.g, a contracting protocol
could look for package size beyond the upper bound anti-Dos limit. Or even the
global ressources offered by the network of full-nodes are not high enough
to handle some application event. E.g a Lightning Service Provider doing a
liquidity maintenance round of all its counterparties, and as such
force-closing and broadcasting more transactions than can be handled at the
transaction-relay layer due to default MAX_PEER_TX_ANNOUNCEMENTS value.
My personal take on those subjects, we might have to realize we're facing
an heterogeneity of Bitcoin applications and use-cases [1]. And this sounds
like a long-term upward trend, akin to the history of the Internet: mail
clients, web browser, streaming applications, etc, all with different
service-level requirements in terms of latency, jitters and bandwidth. We might
observe that Bitcoin actors might adopt individual strategies for the
consumption of it. E.g a multi-party contracting protocol with asymmetric
states, a counterparty could "front-run" the others at the
transaction-relay to avoid encumbering the RBF penalty in case of
concurrent broadcast. Or they could deploy an additive layer of
communication channels, like private transaction-relay to miners, e.g a LN
node broadcasting a HTLC-preimage and restraining the odds of mapping to an
off-chain payment path. Those additive layers of communication can be also
seen as mitigations against components failure risks, e.g a
transaction-relay
censorship vector in the p2p stack. To put it simply, some advanced Bitcoin
applications might have to outgrow the "mempool policy rules" game, as
we're currently defining it to satisfy to a higher degree their security
and operational models. I think this has been historically the case with
some miners deciding to join FIBER, to improve their view of mined blocks.
While I still think we can come up with clear policy rules for some ideal
or reasonable use-case (let's say the Bitcoin hobbyist aiming to strong
censorship-resistance of its
payment). And I hold the belief we can improve on that front, as we've done
during the past years by formalizing a subset of policy rules in Bitcoin
Core documentation. In
parallel, we might acknowledge the heterogeneity of use-cases, and that we
cannot offer the same level of censorship-resistance, privacy or
cost-effectiveness (And that might be okay if we remember that e.g
censorship-resistance between a simple Bitcoin payment and a Lightning HTLC
isn't defined the same, due to the usage of timelocks by the latter).
What I'm expressing is a long-term perspective, and we might be too early
in the evolutionary process that is Bitcoin Core development to abandon yet
the "one-size-fits-all"
policy rules conception that I understand from your post. Though to me, as
we progress on modularity and flexibility of the Core codebase, we might
have to envision a Bitcoin ecosystem where configuring a client is on the
same bar as the kernel Kconfig (hopefully not because no drivers/ but
still..). Of course, you can have "blessed" config to avoid the complexity
turning away node operators and users, though at the price of increasing
the trust in your vendors.
More concretely, about the issue of today, full-rbf, I think the lack of
design heuristic in the sense of what should be the reasonable or average
use-case
supported by the Bitcoin base-layer transaction-relay rules. Should we
favor the historical ones (0confs) or the new incumbent (contracting
protocols) ? There is always the "First, do not harm" principle in lack of
clear guidelines in any area of Bitcoin protocol development. Though when
we're facing risk arbitrage like the current one, I don't think as protocol
devs we have any strong
Re: [bitcoin-dev] On mempool policy consistency
During off-channel discussion, Suhas made a great point that even with fullrbf, you can get stuck by bip125 rule#5 pinning if an adversary controls a number of inputs(4 with default mempool settings). Implication being, while we can mitigate rule#3 damage potentially with fullrbf, we cannot actually make promises about mempool entry beyond quite small transaction sizes. Adversary has to make 100 transactions, 4 chains of 25, but it achieves the original pin. On Thu, Oct 27, 2022 at 1:44 PM Greg Sanders wrote: > > For instance, the double-spend could be low-feerate and large, and > effectively pin any attempt to replace it. > > Yes, this is the biggest hole left. You *could* replace it with RBF when > before you simply could not, so perhaps the pinning door is slightly > smaller in scenarios where going feerates are significantly higher than min. > > > Or it could be higher feerate and confirm and B/C have to start all over. > > Coinjoins have "blame rounds" exactly for this. Ruling out the above hole > where you don't want to pay the 100kvb rule#3 penalty, you can kick the > griefer out. Without replacement, you likely can not. > > > Or, A could stall things in the signing phase and B/C have to figure out > when to give up on the channel with A. > > Again, blame rounds solve this. > > So to recap, it makes it *possible* to over-bid your griefer, vs simply > not able to and have funds tied up for weeks(or guess you're being pinned > and double-spend your input, which again looks blame-worthy). > > Properly replacing rule#3 would give these protocols higher assurances, > but this is where we're at now. > > On Thu, Oct 27, 2022 at 1:35 PM Suhas Daftuar via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> I have more to say on this broader topic, but since you've brought up >> this particular example I think it's worth commenting: >> >> On Thu, Oct 27, 2022 at 1:23 PM Anthony Towns via bitcoin-dev < >> bitcoin-dev@lists.linuxfoundation.org> wrote: >> >>> Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid >>> a DoS issue when utxos are jointly funded by untrusting partners, and, >>> aiui, that's the main motivation for addressing this now. >>> >>> [1] >>> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html >>> >>> The scenario he describes is: A, B, C create a tx: >>> >>> inputs: A1, B1, C1 [opts in to RBF] >>> fees: normal >>> outputs: >>> [lightning channel, DLC, etc, who knows] >>> >>> they all analyse the tx, and agree it looks great; however just before >>> publishing it, A spams the network with an alternative tx, double >>> spending her input: >>> >>> inputs: A1 [does not opt in to RBF] >>> fees: low >>> outputs: A >>> >>> If A gets the timing right, that's bad for B and C because they've >>> populated their mempool with the 1st transaction, while everyone else >>> sees the 2nd one instead; and neither tx will replace the other. B and >>> C can't know that they should just cancel their transaction, eg: >>> >>> inputs: B1, C1 [opts in to RBF] >>> fees: 50% above normal >>> outputs: >>> [smaller channel, refund, whatever] >>> >>> and might instead waste time trying to fee bump the tx to get it mined, >>> or similar. >>> >>> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to >>> solve that problem if they have only opt-in RBF available? >>> >> >> I think this is not a real example of a DoS vector that is available >> because we support non-rbf signaling transactions. Even in a world where >> all transactions are replaceable, person A could double-spend their input >> in a way that is annoying for B and C. For instance, the double-spend >> could be low-feerate and large, and effectively pin any attempt to replace >> it. Or it could be higher feerate and confirm and B/C have to start all >> over. Or, A could stall things in the signing phase and B/C have to figure >> out when to give up on the channel with A. >> >> So I find this example to be unconvincing. Are there any other examples >> where having a non-replacement policy for some transactions causes problems >> for protocols people are trying to build? >> >> Thanks, >> Suhas >> ___ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
More generally, some of the arguments against full RBF seem like debatable reasons (though not fully convincing) to possibly leave it off, and/or disabled by default, but definitely NOT reasons to remove the option and prevent users from deciding for themselves. On Thursday 27 October 2022 15:37:27 Anthony Towns via bitcoin-dev wrote: > "Can I prevent someone else's transaction from propagating" is almost > the entirety of the question with -datacarrier, -datacarriersize and > -permitbaremultisig though: Not necessarily the entirety, no. Even if others would propagate it, you also don't want to waste _your_ bandwidth doing so. This also reveals a difference between the two policies: with RBF, you have _already_ spent resources propagating the first transaction (what this implies is not immediately obvious). Luke ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
> For instance, the double-spend could be low-feerate and large, and effectively pin any attempt to replace it. Yes, this is the biggest hole left. You *could* replace it with RBF when before you simply could not, so perhaps the pinning door is slightly smaller in scenarios where going feerates are significantly higher than min. > Or it could be higher feerate and confirm and B/C have to start all over. Coinjoins have "blame rounds" exactly for this. Ruling out the above hole where you don't want to pay the 100kvb rule#3 penalty, you can kick the griefer out. Without replacement, you likely can not. > Or, A could stall things in the signing phase and B/C have to figure out when to give up on the channel with A. Again, blame rounds solve this. So to recap, it makes it *possible* to over-bid your griefer, vs simply not able to and have funds tied up for weeks(or guess you're being pinned and double-spend your input, which again looks blame-worthy). Properly replacing rule#3 would give these protocols higher assurances, but this is where we're at now. On Thu, Oct 27, 2022 at 1:35 PM Suhas Daftuar via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > I have more to say on this broader topic, but since you've brought up this > particular example I think it's worth commenting: > > On Thu, Oct 27, 2022 at 1:23 PM Anthony Towns via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid >> a DoS issue when utxos are jointly funded by untrusting partners, and, >> aiui, that's the main motivation for addressing this now. >> >> [1] >> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html >> >> The scenario he describes is: A, B, C create a tx: >> >> inputs: A1, B1, C1 [opts in to RBF] >> fees: normal >> outputs: >> [lightning channel, DLC, etc, who knows] >> >> they all analyse the tx, and agree it looks great; however just before >> publishing it, A spams the network with an alternative tx, double >> spending her input: >> >> inputs: A1 [does not opt in to RBF] >> fees: low >> outputs: A >> >> If A gets the timing right, that's bad for B and C because they've >> populated their mempool with the 1st transaction, while everyone else >> sees the 2nd one instead; and neither tx will replace the other. B and >> C can't know that they should just cancel their transaction, eg: >> >> inputs: B1, C1 [opts in to RBF] >> fees: 50% above normal >> outputs: >> [smaller channel, refund, whatever] >> >> and might instead waste time trying to fee bump the tx to get it mined, >> or similar. >> >> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to >> solve that problem if they have only opt-in RBF available? >> > > I think this is not a real example of a DoS vector that is available > because we support non-rbf signaling transactions. Even in a world where > all transactions are replaceable, person A could double-spend their input > in a way that is annoying for B and C. For instance, the double-spend > could be low-feerate and large, and effectively pin any attempt to replace > it. Or it could be higher feerate and confirm and B/C have to start all > over. Or, A could stall things in the signing phase and B/C have to figure > out when to give up on the channel with A. > > So I find this example to be unconvincing. Are there any other examples > where having a non-replacement policy for some transactions causes problems > for protocols people are trying to build? > > Thanks, > Suhas > ___ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
I have more to say on this broader topic, but since you've brought up this particular example I think it's worth commenting: On Thu, Oct 27, 2022 at 1:23 PM Anthony Towns via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid > a DoS issue when utxos are jointly funded by untrusting partners, and, > aiui, that's the main motivation for addressing this now. > > [1] > https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html > > The scenario he describes is: A, B, C create a tx: > > inputs: A1, B1, C1 [opts in to RBF] > fees: normal > outputs: > [lightning channel, DLC, etc, who knows] > > they all analyse the tx, and agree it looks great; however just before > publishing it, A spams the network with an alternative tx, double > spending her input: > > inputs: A1 [does not opt in to RBF] > fees: low > outputs: A > > If A gets the timing right, that's bad for B and C because they've > populated their mempool with the 1st transaction, while everyone else > sees the 2nd one instead; and neither tx will replace the other. B and > C can't know that they should just cancel their transaction, eg: > > inputs: B1, C1 [opts in to RBF] > fees: 50% above normal > outputs: > [smaller channel, refund, whatever] > > and might instead waste time trying to fee bump the tx to get it mined, > or similar. > > What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to > solve that problem if they have only opt-in RBF available? > I think this is not a real example of a DoS vector that is available because we support non-rbf signaling transactions. Even in a world where all transactions are replaceable, person A could double-spend their input in a way that is annoying for B and C. For instance, the double-spend could be low-feerate and large, and effectively pin any attempt to replace it. Or it could be higher feerate and confirm and B/C have to start all over. Or, A could stall things in the signing phase and B/C have to figure out when to give up on the channel with A. So I find this example to be unconvincing. Are there any other examples where having a non-replacement policy for some transactions causes problems for protocols people are trying to build? Thanks, Suhas ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
On Thu, Oct 27, 2022 at 11:56:45AM +0200, John Carvalho via bitcoin-dev wrote: > I took the time to read your whole post. Despite a diplomatic tone, I find > your takeaways from all your references to remain conveniently biased for > protecting the plan of RBF Yes, I am heavily biased against zeroconf: there's no way I'd personally be willing to trust it for my own incoming funds, no matter how much evidence you show me that it's safe in practice. Show me a million transactions where every single one worked fine, and I'm still going to assume that the payment going to me is going to be the one that makes the error rate tick up from 0% to 0.0001%. That's okay; just because I wouldn't do something, doesn't mean other people shouldn't. It does mean I'm not going to be a particularly good advocate for zeroconf though. I mean, I might still be a fine advocate for giving people time to react, making it clear what's going on, finding ways that might make everyone happy, or just digging it to random technical details; but, for me, I'm more interested in a world where chargebacks are impossible, not where we just make the best of what was possible with technology from five or ten years ago. But that's fine: it just means that people, like yourself, who will tolerate the risks of zeroconf, should be involved in the discussion. > You show multiple examples where, when I read them, I assume the next thing > you will say will be "so we really should stop trying to impose optional > features, particularly when they affect existing use cases" but instead you > persist. Sure, that's natural: you read a sign saying "you can have any ice cream you want for 5c" and think "Awesome, who wouldn't want cheap chocolate ice cream!!" and see me going for a Golden Gaytime and think "wtf dude". Different strokes. For me, I see the gmaxwell github comment I quoted saying: There is also a matter of driving competent design rather than lazy first thing that works. and think "yeah, okay, maybe we should be working harder to push lightning adoption, rather than letting people stick with wallet UX from 2015" and have altcoins take over >50% of payment volume. Likewise, There is also a very clear pattern we've seen in the past where people take anything the system lets them do as strong evidence that they have a irrevocable right to use the system in that way, and that their only responsibility-- and if their usage harms the system it's the responsibility of the system to not permit it. seems a pretty good match against your claim "I expect the things I do with Bitcoin today to work FOREVER." Better to nip that thinking in the bud; and even if the best time to do that was years ago, the second best time to do it is still now. By contrast, from the same post, I'd guess you're focussing on: Network behavior is one of the few bits of friction driving good technical design rather than "move fast, break things, and force everyone else onto my way of doing thing rather than discussing the design in public". and thinking "yeah, move fast, break things, force everyone else -- that's exactly what's going on here, and shouldn't be". But that's also okay: even when there is common ground to be found, sometimes it requires actual work to get people who start from different views to get there. > The problem is that RBF has already been an option for years, and anyone > that wants to use it can. Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid a DoS issue when utxos are jointly funded by untrusting partners, and, aiui, that's the main motivation for addressing this now. [1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html The scenario he describes is: A, B, C create a tx: inputs: A1, B1, C1 [opts in to RBF] fees: normal outputs: [lightning channel, DLC, etc, who knows] they all analyse the tx, and agree it looks great; however just before publishing it, A spams the network with an alternative tx, double spending her input: inputs: A1 [does not opt in to RBF] fees: low outputs: A If A gets the timing right, that's bad for B and C because they've populated their mempool with the 1st transaction, while everyone else sees the 2nd one instead; and neither tx will replace the other. B and C can't know that they should just cancel their transaction, eg: inputs: B1, C1 [opts in to RBF] fees: 50% above normal outputs: [smaller channel, refund, whatever] and might instead waste time trying to fee bump the tx to get it mined, or similar. What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to solve that problem if they have only opt-in RBF available? If you're right that opt-in RBF is enough, that question has a good answer. I don't believe anyone's presented an answer to it in the 17 months since Antoine raised the concern. > passive aggression > escalation > unfair advantage > oppressive, dark-pattern design >
Re: [bitcoin-dev] On mempool policy consistency
On Thu, Oct 27, 2022 at 01:36:47PM +0100, Gloria Zhao wrote:
> > The cutoff for that is probably something like "do 30% of listening
> > nodes have a compatible policy"? If they do, then you'll have about a
> > 95% chance of having at least one of your outbound peers accept your tx,
> > just by random chance.
> Yes, in most cases, whether Bitcoin Core is restricting or loosening
> policy, the user in question is fine as long as they have a path from their
> node to a miner that will accept it. This is the case for something like
> -datacarriersize if the use case is putting stuff into OP_RETURN outputs,
> or if they're LN and using CPFP carveout, v3, package relay, etc.
> But
> replacement is not only a question of "will my transaction propagate" but
> also, "will someone else's transaction propagate, invalidating mine" or, in
> other words, "can I prevent someone else's transaction from propagating."
"Can I prevent someone else's transaction from propagating" is almost
the entirety of the question with -datacarrier, -datacarriersize and
-permitbaremultisig though: "we" don't want people to spam the utxo
set or the blockchain with lots of data (cf BSV's gigabytes worth of
dog pictures [0]), so for the people who are going to find some way
of putting data in we'd like to encourage them to make it small, and
do it in a way that's prunable and doesn't bloat the utxo set, whose
size matters even more than the overall blockchain's size does. As I
understand it, people were doing that by creating bare multisig utxos,
ie a bare (non-p2sh) scriptPubKey that perhaps looks like:
1 my_key data1 data2 data3 data4 data5 5 CHECKMULTISIG
which is "bad" in two ways: you're only committing to the data, so why
not save 128 bytes by doing hash(data1 data2 data3 data4 data5) instead;
and even more so, that data is only interesting to you, not everyone else,
so why not do it in a way that doesn't bloat the utxo set, which we want
to keep small so that it's easier to efficiently look up potential spends.
Hence the -datacarriersize limitation that limits you to about 2.5
"dataN" entries per tx ("we'll prevent your tx from propagating if you
do much more than publish a hash") and hence at least the potential for
doing the same for baremultisig in general.
[0] https://twitter.com/bsvdata/status/1427866510035324936
> A
> zeroconf user relies on there *not* being a path from someone else's full
> RBF node to a full RBF miner. This is why I think RBF is so controversial
> in general,
Yes; but I think it's also true to say that this is why *zeroconf* is as
controversial as it is.
Likewise OP_RETURN has had its own "controversies" to some extent, too:
https://blog.bitmex.com/dapps-or-only-bitcoin-transactions-the-2014-debate/
https://github.com/bitcoin/bitcoin/pull/3715
https://github.com/bitcoin/bitcoin/pull/3939
> why -mempoolfullrbf on someone else's node is considered more
> significant than another policy option, and why full RBF shouldn't be
> compared with something like datacarriersize.
It's definitely a different scenario: unexpected RBF can cause you to
have less money than you expected; whereas more OP_RETURN data just
bloats the blockchain, and losing money that you thought was yours is
definitely more painful than more spam.
But while the level of pain is different; I don't think the mechanism is:
whether you're trying to preserve zeroconf or prevent utxo set spam,
you're still relying on a vast majority of nodes working together to
prevent even a small minority of hashpower from doing "bad" things, with
no cryptographic assurances that will continue to work well or at all.
> I don't think past patterns can be easily applied here,
I mean, technically they trivially could? We *could* roll out support for
full RBF in exactly the same way we rolled out support for opt-in RBF:
making it the default for all nodes, but supplying an option that node
operators can use to disable the feature for seven releases / ~4 years:
https://bitcoin.org/en/release/v0.12.0#opt-in-replace-by-fee-transactions
https://bitcoin.org/en/release/v0.19.0.1#deprecated-or-removed-configuration-options
If we don't want to do that immediately, but also want to make a definite
move forward, then we could:
* just say that, and then keep our word about it
* keep the feature in master, but remove it in 24.x
* put a time delay on the feature so that it doesn't happen immediately
but is locked in in the code for whenever we are ready to do it
> and I don't think this necessarily shows a
> different "direction" in thinking about mempool policy in general.
If we're not applying past patterns, then this is a different direction
in how we're thinking about things than what we did in the past.
That's not necessarily a bad thing -- maybe we should be thinking
differently; but I don't see how you can honestly dispute it: those are
just two ways of saying the exact same thing.
Cheers,
aj
___
Re: [bitcoin-dev] On mempool policy consistency
On Thu, Oct 27, 2022 at 09:49:48AM -0400, Greg Sanders via bitcoin-dev wrote: > So there is some precedence to including an option that protocol devs don't > find useful, then removing it N years later to make sure it doesn't impact > compact blocks. I think the lesson there is we're willing to remove options that are ridiculous. Replacements are widely used, and downright essential in high-fee situations. > Peering into the "precedence" lense, I think this does lend itself to the > theory that the transition should be as uniform as possible to avoid > degradation of fast block propagation. If not removing options(which is > deemed user hostile by a number of folks including me), then at least for a > flag day switchover. Re: compact blocks, note that RBF is a special case: for the sake of reconstruction, it'd make sense to temporarily cache transactions have have been replaced rather than discarding them entirely, in case a prior version gets mined. Irregardless of policy this will happen occasionally simple due to propagation delays. Equally, if we cached transactions that we rejected due to policy, that'd help with reconstruction success in the event that policy is changing. Anyway, since the compact blocks implementation efficiently deals with the case where miners have policy that differs from most nodes, by immediately forwarding missing transactions, I don't think the occasional full-rbf replacement is going to have much impact. The nodes that had full-rbf disabled will forward the tx to their peers directly, and then the subset of full-rbf disabled peers will do the same again. So long as the network has a mix of both types, and they're interconnected rather than in clusters, the latency impact should be minimal. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] On mempool policy consistency
To add a wrinkle, or possibly a confirmation of your long message, up to
readers to decipher, there historically has been at least one more RBF
related option that was included, then removed later in Core.
Introduced as "permitrbf" in b768108d9c0b83330572711aef1e569543130d5e with
default "true", later renamed to "replacebyfee", then finally as
"mempoolreplacement".
It was later removed in 8053e5cdade87550f0381d51feab81dedfec6c46 with the
message:
"
Remove -mempoolreplacement to prevent needless block prop slowness.
At this point there is no reasonable excuse to disable opt-in RBF,
and, unlike when this option was added, there are now significant
issues created when disabling it (in the form of compact block
reconstruction failures). Further, it breaks a lot of modern wallet
behavior.
"
So there is some precedence to including an option that protocol devs don't
find useful, then removing it N years later to make sure it doesn't impact
compact blocks.
Peering into the "precedence" lense, I think this does lend itself to the
theory that the transition should be as uniform as possible to avoid
degradation of fast block propagation. If not removing options(which is
deemed user hostile by a number of folks including me), then at least for a
flag day switchover.
Cheers,
Greg
On Wed, Oct 26, 2022 at 7:53 PM Anthony Towns via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:
> Hi *,
>
> TLDR: Yes, this post is too long, and there's no TLDR. If it's any
> consolation, it took longer to write than it does to read?
>
> On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev
> wrote:
> > Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0
> > I'm writing to propose deprecation of opt-in RBF in favor of full-RBF
>
> > If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds
> > too early, let's defer it to 0.25 or 0.26. I don't think Core has a
> > consistent deprecation process w.r.t to policy rules heavily relied-on by
> > Bitcoin users, if we do so let sets a precedent satisfying as many folks
> as
> > we can.
>
> One precedent that seems to be being set here, which to me seems fairly
> novel for bitcoin core, is that we're about to start supporting and
> encouraging nodes to have meaningfully different mempool policies. From
> what I've seen, the baseline expectation has always been that while
> certainly mempools can and will differ, policies will be largely the same:
>
> Firstly, there is no "the mempool". There is no global mempool. Rather
> each node maintains its own mempool and accepts and rejects transaction
> to that mempool using their own internal policies. Most nodes have
> the same policies, but due to different start times, relay delays,
> and other factors, not every node has the same mempool, although they
> may be very similar.
>
> -
> https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting
>
> Up until now, the differences between node policies supported by different
> nodes running core have been quite small, with essentially the following
> options available:
>
> -minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
>
> -mempoolexpiry - how long to keep txs in the mempool
>
> -datacarrier - reject txs creating OP_RETURN outputs
>
> -datacarriersize - maximum size of OP_RETURN data
>
> -permitbaremultisig - prevent relay of bare multisig
>
> -bytespersigop - changes how SIGOP accounting works for relay and
> mining prioritisation
>
> as well as these, marked as "debug only" options (only shown with
> -help-debug):
>
> -incrementalrelayfee - make it easier/harder to spam txs by only
> slightly bumping the fee; marked as a "debug only" option
>
> -dustrelayfee - make it easier/harder to create uneconomic utxos;
> marked as a "debug only" option
>
> -limit{descendant,ancestor}{count,size} - changes how large the
> transaction chains can be; marked as a "debug only" option
>
> and in theory, but not available on mainnet:
>
> -acceptnonstdtxn - relay/mine non standard transactions
>
> There's also the "prioritisetransaction" rpc, which can cause you to keep
> a low feerate transaction in your mempool longer than you might otherwise.
>
> I think that -minrelaytxfee, -maxmempool and -mempoolexpiry are the only
> ones of those options commonly set, and those only rarely result in any
> differences in the txs at the top of the mempool.
>
> There are also quite a few parameters that aren't even runtime
> configurable:
>
> - MAX_STANDARD_TX_WEIGHT
> - MIN_STANDARD_TX_NONWITNESS_SIZE (see also #26265)
> - MAX_P2SH_SIGOPS (see also #26348)
> - MAX_STANDARD_TX_SIGOPS_COST
> - MAX_STANDARD_P2WSH_STACK_ITEMS
> - MAX_STANDARD_P2WSH_STACK_ITEM_SIZE
> - MAX_STANDARD_TAPSCRIPT_STACK_ITEM_SIZE
> - MAX_STANDARD_P2WSH_SCRIPT_SIZE
> - MAX_STANDARD_SCRIPTSIG_SIZE
> - EXTRA_DESCENDANT_TX_SIZE_LIMIT
> - MAX_REPLACEMENT_CANDIDATES
Re: [bitcoin-dev] On mempool policy consistency
Hi AJ,
Not going to comment on what Bitcoin Core's philosophy on mempol policy is
or should be. I want to note that I think this:
> It's also possible that this is something of a one time thing: full rbf
> has been controversial for ages, but widely liked by devs, and other
> attempts (eg making it available in knots) haven't actually achieved
> much of a result in practice. So maybe this is just a special case
is true.
> The second thing is that whatever your relay policy is, you still
> need a path all the way to miners through nodes that will accept your
> transaction at every step. If you're making your mempool more restrictive
> (eg -permitbaremultisig=0, -datacarrier=0), that's easy for you (though
> you're making life more difficult for people who do create those sorts
> of txs); but if you want a more permissive policy (package relay,
> version-3-rbf, full-rbf), you might need to do some work.
> The cutoff for that is probably something like "do 30% of listening
> nodes have a compatible policy"? If they do, then you'll have about a
> 95% chance of having at least one of your outbound peers accept your tx,
> just by random chance.
Yes, in most cases, whether Bitcoin Core is restricting or loosening
policy, the user in question is fine as long as they have a path from their
node to a miner that will accept it. This is the case for something like
-datacarriersize if the use case is putting stuff into OP_RETURN outputs,
or if they're LN and using CPFP carveout, v3, package relay, etc. But
replacement is not only a question of "will my transaction propagate" but
also, "will someone else's transaction propagate, invalidating mine" or, in
other words, "can I prevent someone else's transaction from propagating." A
zeroconf user relies on there *not* being a path from someone else's full
RBF node to a full RBF miner. This is why I think RBF is so controversial
in general, why -mempoolfullrbf on someone else's node is considered more
significant than another policy option, and why full RBF shouldn't be
compared with something like datacarriersize. I don't think past patterns
can be easily applied here, and I don't think this necessarily shows a
different "direction" in thinking about mempool policy in general.
Best,
Gloria
On Thu, Oct 27, 2022 at 12:52 AM Anthony Towns via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:
> Hi *,
>
> TLDR: Yes, this post is too long, and there's no TLDR. If it's any
> consolation, it took longer to write than it does to read?
>
> On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev
> wrote:
> > Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0
> > I'm writing to propose deprecation of opt-in RBF in favor of full-RBF
>
> > If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds
> > too early, let's defer it to 0.25 or 0.26. I don't think Core has a
> > consistent deprecation process w.r.t to policy rules heavily relied-on by
> > Bitcoin users, if we do so let sets a precedent satisfying as many folks
> as
> > we can.
>
> One precedent that seems to be being set here, which to me seems fairly
> novel for bitcoin core, is that we're about to start supporting and
> encouraging nodes to have meaningfully different mempool policies. From
> what I've seen, the baseline expectation has always been that while
> certainly mempools can and will differ, policies will be largely the same:
>
> Firstly, there is no "the mempool". There is no global mempool. Rather
> each node maintains its own mempool and accepts and rejects transaction
> to that mempool using their own internal policies. Most nodes have
> the same policies, but due to different start times, relay delays,
> and other factors, not every node has the same mempool, although they
> may be very similar.
>
> -
> https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting
>
> Up until now, the differences between node policies supported by different
> nodes running core have been quite small, with essentially the following
> options available:
>
> -minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
>
> -mempoolexpiry - how long to keep txs in the mempool
>
> -datacarrier - reject txs creating OP_RETURN outputs
>
> -datacarriersize - maximum size of OP_RETURN data
>
> -permitbaremultisig - prevent relay of bare multisig
>
> -bytespersigop - changes how SIGOP accounting works for relay and
> mining prioritisation
>
> as well as these, marked as "debug only" options (only shown with
> -help-debug):
>
> -incrementalrelayfee - make it easier/harder to spam txs by only
> slightly bumping the fee; marked as a "debug only" option
>
> -dustrelayfee - make it easier/harder to create uneconomic utxos;
> marked as a "debug only" option
>
> -limit{descendant,ancestor}{count,size} - changes how large the
> transaction chains can be; marked as a "debug only" option
>
> and in
Re: [bitcoin-dev] On mempool policy consistency
Anthony, I took the time to read your whole post. Despite a diplomatic tone, I find your takeaways from all your references to remain conveniently biased for protecting the plan of RBF via passive aggression. You show multiple examples where, when I read them, I assume the next thing you will say will be "so we really should stop trying to impose optional features, particularly when they affect existing use cases" but instead you persist. The problem is that RBF has already been an option for years, and anyone that wants to use it can. Any escalation in Bitcoin Core code to support it more deeply, or by default, is basically an unfair advantage to force the market to do what it already has decided not to. If wallets want to default to RBF, they can already do so, as evidenced by Green Wallet (which I stopped using because it breaks the UX at Bitrefill). Instead of Core devs admitting RBF is a minority use case, you seem to be proposing that the market should now be obligated to prove it can defeat RBF in a stronger form if it really wants to prove other use cases. This is oppressive, dark-pattern design. We all know that Core has little ability to sense the market, and the market has little ability to express itself to Core. The idea that the market can always downvote or defeat a feature or new complexity proposal is idealistic and unrealistic. Superficial features should be decided at the surface (app level) not in the protocol or node. The default answer to ALL proposals is "No." Changes need to win market acceptance, not get special access through Core devs baking them deeper and deeper into the protocol and policies until everyone is forced into a new design. As I mentioned before, this behavior, if we are lucky, will result in more mempool types, more implementations, and a more-difficult to modify protocol, but ALL feature changes, default settings that make decisions for users, and even all scaling changes, are speculative risks with unpredictable outcomes. I urge the culture of Core to respect these dynamics and become much more conservative with proposing change. Please focus on efficiencies, bugs, cleanup, reducing overhead, etc. The current RBF movement feels like Core is strong-arming and shoe-horning in a change that the market is not actually asking for. It is okay to leave things as they are. It is okay if RBF remains a niche feature. It is not okay for a small group of RBF-interested engineers to make commercial Bitcoin use cases worse. Let us realize the Bitcoin we already have. We already have a largely unexplored canvas of taproot, lightning, UX, etc. I expect the things I do with Bitcoin today to work FOREVER. -- John Carvalho CEO, Synonym.to <http://synonym.to/> > Date: Thu, 27 Oct 2022 09:52:10 +1000 > From: Anthony Towns > To: Bitcoin Protocol Discussion > > Subject: [bitcoin-dev] On mempool policy consistency > Message-ID: > Content-Type: text/plain; charset=us-ascii > > Hi *, > > TLDR: Yes, this post is too long, and there's no TLDR. If it's any > consolation, it took longer to write than it does to read? > > On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev > wrote: > > Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0 > > I'm writing to propose deprecation of opt-in RBF in favor of full-RBF > > > If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds > > too early, let's defer it to 0.25 or 0.26. I don't think Core has a > > consistent deprecation process w.r.t to policy rules heavily relied-on by > > Bitcoin users, if we do so let sets a precedent satisfying as many folks > as > > we can. > > One precedent that seems to be being set here, which to me seems fairly > novel for bitcoin core, is that we're about to start supporting and > encouraging nodes to have meaningfully different mempool policies. From > what I've seen, the baseline expectation has always been that while > certainly mempools can and will differ, policies will be largely the same: > > Firstly, there is no "the mempool". There is no global mempool. Rather > each node maintains its own mempool and accepts and rejects transaction > to that mempool using their own internal policies. Most nodes have > the same policies, but due to different start times, relay delays, > and other factors, not every node has the same mempool, although they > may be very similar. > > - > https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting > > Up until now, the differences between node policies supported by different > nodes running core have been quite small, with essentially the following > options available: > > -minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
[bitcoin-dev] On mempool policy consistency
Hi *,
TLDR: Yes, this post is too long, and there's no TLDR. If it's any
consolation, it took longer to write than it does to read?
On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev wrote:
> Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0
> I'm writing to propose deprecation of opt-in RBF in favor of full-RBF
> If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds
> too early, let's defer it to 0.25 or 0.26. I don't think Core has a
> consistent deprecation process w.r.t to policy rules heavily relied-on by
> Bitcoin users, if we do so let sets a precedent satisfying as many folks as
> we can.
One precedent that seems to be being set here, which to me seems fairly
novel for bitcoin core, is that we're about to start supporting and
encouraging nodes to have meaningfully different mempool policies. From
what I've seen, the baseline expectation has always been that while
certainly mempools can and will differ, policies will be largely the same:
Firstly, there is no "the mempool". There is no global mempool. Rather
each node maintains its own mempool and accepts and rejects transaction
to that mempool using their own internal policies. Most nodes have
the same policies, but due to different start times, relay delays,
and other factors, not every node has the same mempool, although they
may be very similar.
-
https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting
Up until now, the differences between node policies supported by different
nodes running core have been quite small, with essentially the following
options available:
-minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
-mempoolexpiry - how long to keep txs in the mempool
-datacarrier - reject txs creating OP_RETURN outputs
-datacarriersize - maximum size of OP_RETURN data
-permitbaremultisig - prevent relay of bare multisig
-bytespersigop - changes how SIGOP accounting works for relay and
mining prioritisation
as well as these, marked as "debug only" options (only shown with
-help-debug):
-incrementalrelayfee - make it easier/harder to spam txs by only
slightly bumping the fee; marked as a "debug only" option
-dustrelayfee - make it easier/harder to create uneconomic utxos;
marked as a "debug only" option
-limit{descendant,ancestor}{count,size} - changes how large the
transaction chains can be; marked as a "debug only" option
and in theory, but not available on mainnet:
-acceptnonstdtxn - relay/mine non standard transactions
There's also the "prioritisetransaction" rpc, which can cause you to keep
a low feerate transaction in your mempool longer than you might otherwise.
I think that -minrelaytxfee, -maxmempool and -mempoolexpiry are the only
ones of those options commonly set, and those only rarely result in any
differences in the txs at the top of the mempool.
There are also quite a few parameters that aren't even runtime
configurable:
- MAX_STANDARD_TX_WEIGHT
- MIN_STANDARD_TX_NONWITNESS_SIZE (see also #26265)
- MAX_P2SH_SIGOPS (see also #26348)
- MAX_STANDARD_TX_SIGOPS_COST
- MAX_STANDARD_P2WSH_STACK_ITEMS
- MAX_STANDARD_P2WSH_STACK_ITEM_SIZE
- MAX_STANDARD_TAPSCRIPT_STACK_ITEM_SIZE
- MAX_STANDARD_P2WSH_SCRIPT_SIZE
- MAX_STANDARD_SCRIPTSIG_SIZE
- EXTRA_DESCENDANT_TX_SIZE_LIMIT
- MAX_REPLACEMENT_CANDIDATES
And other plausible options aren't configurable even at compile time
-- eg, core doesn't implement BIP 125's inherited signalling rule so
there's no way to enable it; core doesn't allow opting out of BIP 125
rule 3 ratchet on absolute fee; core doesn't allow CPFP carveout with
more than 1 ancestor; core doesn't allow opting out of LOW_S checks
(even via -acceptnonstdtxn); etc.
We also naturally have different mempool policies between different
releases: eg, expansions of policy, such as allowing OP_RETURN or
expanding it from 40 to 80 bytes or new soft forks where old nodes won't
relay transactions that use the new; and also occassional restrictions
in policy, such as the LOW_S requirement.
While supporting and encouraging different mempool polices might be new
for core, it's not new for knots: knots changes some of these defaults
(-permitbaremultisig defaults to false, -datacarriersize is reduced to
42), allows the use of -acceptnonstdtxn on mainnet, and introduces new
options including -spkreuse and -mempoolreplacement (giving the latter
full rbf behaviour by default). Knots also includes a `-corepolicy`
option to make it easy to get a configuration matching core's defaults.
I think gmaxwell's take from Feb 2015 (in the context of how restrictive
policy on OP_RETURN data should be) was a reasonable description for
core's approach up until now:
There is also a matter of driving competent design rather than lazy
first thing that works. E.g. In stealth addresses the early proposals
use highly inefficient single ECDH point per output instead
