Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
> IIRC correctly, in your scenario, you're dealing with Carol -> Bob -> Alice. > Mallory can only replace an HTLC-timeout transaction if she's directly > connected with the peer she's targeting via a direct channel. She cannot > unilaterally replace any transaction in the mempool solely with knowledge of > the preimage. All HTLC transactions are two party contracts, with the public > keys of both participants hard coded into the contract. A third party cannot > unilaterally sweep an HTLC given only a preimage. > I think it's worth taking a minute to step back here so we can establish > some fundamental context. Once the timelock of an HTLC expires, and the > receiver has direct knowledge of the preimage, a bidding war begins. If the > receiver is able to confirm their success transaction in time, then they > gain the funds, and the sender is able to pipeline the preimage backwards in > the route, making all parties whole. If the sender is the one that wins out, > then it's as if nothing happened, sans the fees paid at the last mile, all > other hops are able to safely cancel their HTLC back on the chain. > As mentioned elsewhere in the thread, most implementations today will watch > the mempool for preimages, so they can resolve the incoming HTLC as quickly > as possible off chain. The attack described critically relies on the ability > of an attacker to pull off very precise transaction replacement globally > across "the mempool". If any of the honest parties see the preimage, in the > mempool, then the attack ends as they can settle back off chain, and if > their timeout is able to confirm first, then the defender has actually > gained funds. Correct mostly so far, a part of the globally across "the mempool". > In addition, such an attack needs to be executed perfectly, for hours if not > days. Within the LN, all nodes set a security parameter, the CLTV delta, > that dictates how much time they have before the outgoing and incoming HTLCs > expire. Increasing this value makes the attack more difficult, as they must > maintain the superposition of: fees low enough to not get mined, but high > enough to replace the defender's transaction, but not _too_ high, as then > it'll be mined and everything is over. With each iteration, the attacker is > also forced to increase the amount of fees they pay, increasing the likely > hood that the transaction will be mined. If mined, the attack is moot. The attacker just has to overbid the parent of the HTLC-preimage with a higher fee replacement parent. This replacement parent can be confirmed at any time, even if the attacker realizes a higher economic gain if this replacement transaction is delayed as much as possible. Or the absolute fee consumed to batch attack multiple targets. So I think your assumption on the attack difficulty is not correct. > As mentioned in the OP, one thing the attacker can do is try and locate the > defender's node to launch the replacement attack directly in their mempool. > However, by replacing directly in the mempool of the defender, the defender > will learn of the preimage! They can use that to just settle everything > back, thereby completing the payment, and stopping the attack short. Even > ignoring direct access to the defender's mempool, the attacker needs to > somehow iteratively execute the replacement across a real network, with all > the network jitter, propagation delay, and geographic heterogeneity that > entails. If they're off by milliseconds, then either a confirmation occurs, > or the preimage is revealed in the mempool. A canned local regtest env is > one thing, the distributed system that is the Internet is another. The attacker can blind the defender mempool by locating the associated full-node with its lightning node, then broadcast a conflicting parent of the HTLC-preimage, partitioning the mempool from the rest of the network. The HTLC-preimage will never enter the defender's mempool, as some spent inputs are missing. All the considerations on network jitter, propagation delay and geographic heterogeneity are irrelevant as network mempools are in practice a distributed system of logically equivalent state machines to favor propagation of high-fees paying transactions, on top of raw TCP/IP connections. Propagation works quite deterministically on average if you have ever used a bitcoin wallet to pay with zero-conf and see the transaction showing up as unconfirmed on the payment processor. Policy divergences across implementations and versions does not matter here. The p2p stack of Bitcoin Core has some propagation delay in the order of seconds for outbound / inbound connections transaction-relay connections. I think they're at the advantage of the attacker, as they can be bypassed to quickly evict the honest HTLC-timeout bypassing them with mass connections. The p2p transaction-relay is a public network of nodes running common open software, where their inbound slots can be occupied at low-cost and their
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On Sat, Oct 21, 2023 at 09:05:35PM +0100, Antoine Riard via bitcoin-dev wrote: > In the meanwhile, lightning experts have already deployed mitigations which > are hardening the lightning ecosystem significantly in face of simple or > medium attacks. More advanced attacks can only be mounted if you have > sufficient p2p and mempool knowledge as was pointed out by other bitcoin > experts like Matt or Peter (which take years to acquire for average bitcoin > developers) and the months of preparation to attempt them. To be clear, I am not making any claims about how easy this attack is to pull off. Indeed, there are probably even cases where it happens by accident. Eg imagine a node with a HTLC-preimage that happens to be offline and then online at the right time to broadcast a HTLC-preimage redemption transaction with a higher fee than the timeout transaction. If the other node happens to go offline at the right time, after broadcasting the timeout transaction, it may not notice the HTLC-preimage in the mempool, and thus fail to redeem it. OP_Expire would help avoid this situation, by making it impossible to redeem the HTLC-preimage after the timeout. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On 10/20/23 7:43 PM, Peter Todd wrote: On Fri, Oct 20, 2023 at 09:55:12PM -0400, Matt Corallo wrote: Quite the contrary. Schnorr signatures are 64 bytes, so in situations like lightning where the transaction form is deterministically derived, signing 100 extra transactions requires just 6400 extra bytes. Even a very slow 100KB/s connection can transfer that in 64ms; latency will still dominate. Lightning today isn't all that much data, but multiply it by 100 and we start racking up data enough that people may start to have to store a really material amount of data for larger nodes and dealing with that starts to be a much bigger pain then when we're talking about a GiB or twenty. We are talking about storing ephemeral data here, HTLC transactions and possibly commitment transactions. Since lightning uses disclosed secrets to invalidate old state, you do not need to keep every signature from your counterparty indefinitely. Mmm, fair point, yes. RBF has a minimum incremental relay fee of 1sat/vByte by default. So if you use those 100 pre-signed transaction variants to do nothing more than sign every possible minimum incremental relay, you've covered a range of 1sat/vByte to 100sat/vByte. I believe that is sufficient to get mined for any block in Bitcoin's entire modern history. CPFP meanwhile requires two transactions, and thus extra bytes. Other than edge cases with very large transactions in low-fee environments, there's no circumstance where CPFP beats RBF. What I was referring to is that if we have the SIGHASH_SINGLE|ANYONECANPAY we can combine many HTLC claims into one transaction, vs pre-signing means we're stuck with a ton of individual transactions. Since SIGHASH_SINGLE requires one output per input, the savings you get by combining multiple SIGHASH_SINGLE transactions together aren't very significant. Just 18 bytes for nVersion, nLockTime, and the txin and txout size fields. The HTLC-timeout transaction is 166.5 vBytes, so that's a savings of just 11% Yep, its not a lot, but for a thing that's inherently super chain-spammy, its still quite nice. Of course, if you _do_ need to fee bump and add an additional input, that input takes up space, and you'll probably need a change output. At which point you again would probably have been better off with a pre-signed transaction. You are also assuming there's lots of HTLC's in flight that need to be spent. That's very often not the case. In general, yes, in force-close cases often there's been some failure which is repeated in several HTLCs :). More generally, I think we're getting lost here - this isn't really a material change for lightning's trust model - its already the case that a peer that is willing to put a lot of work in can probably steal your money, and there's now just one more way they can do that. We really don't need to rush to "fix lightning" here, we can do it right and fix it at the ecosystem level. It shouldn't be the case that a policy restriction results in both screwing up a L2 network *and* results in miners getting paid less. That's a policy bug. Matt ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On 2023-10-21 18:49, Nadav Ivgi via bitcoin-dev wrote: Could this be addressed with an OP_CSV_ALLINPUTS, a covenant opcode that requires _all_ inputs to have a matching nSequence, and using `1 OP_CSV_ALLINPUTS` in the HTLC preimage branch? This would prevent using unconfirmed outputs in the HTLC-preimage-spending transaction entirely, which IIUC should protect it against the replacement cycling attack. I don't think that addresses the underlying problem. In Riard's description, a replacement cycle looks like this: - Bob broadcasts an HTLC-timeout (input A, input B for fees, output X) - Mallory replaces the HTLC-timeout with an HTLC-preimage (input A, input C for fees, output Y) - Mallory replaces the transaction that created input C, removing the HTLC-preimage from the mempool However, an alternative approach is: - (Same) Bob broadcasts an HTLC-timeout (input A, input B for fees, output X) - (Same) Mallory replaces the HTLC-timeout with an HTLC-preimage (input A, input C for fees, output Y) - (Different) Mallory uses input C to replace the HTLC-preimage with a transaction that does not include input A, removing the preimage from the mempool The original scenario requires input C to be from an unconfirmed transaction, so OP_CSV_ALLINPUTS works. The alternative scenario works even if input C comes from a confirmed transaction, so OP_CSV_ALLINPUTS is ineffective. -Dave ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On Fri, Oct 20, 2023 at 09:55:12PM -0400, Matt Corallo wrote: > > Quite the contrary. Schnorr signatures are 64 bytes, so in situations like > > lightning where the transaction form is deterministically derived, signing > > 100 > > extra transactions requires just 6400 extra bytes. Even a very slow 100KB/s > > connection can transfer that in 64ms; latency will still dominate. > > Lightning today isn't all that much data, but multiply it by 100 and we > start racking up data enough that people may start to have to store a really > material amount of data for larger nodes and dealing with that starts to be > a much bigger pain then when we're talking about a GiB or twenty. We are talking about storing ephemeral data here, HTLC transactions and possibly commitment transactions. Since lightning uses disclosed secrets to invalidate old state, you do not need to keep every signature from your counterparty indefinitely. Per channel, with the above numbers, you'd need <10KB worth of extra signatures to fee bump the current commitment (with pre-signed txs), and if HTLCs happen to be in flight, say <10KB of extra signatures per HTLC. Amazon AWS charges $0.125/GB/month for their most expensive SSD volumes. Let's round that up to $1/GB/month to account for RAID and backups. Let's say each channel constantly has 483 HTLC's in flight, and each HTLC is associated with ~10KB of extra data for pre-signed transactions. That's ~5MB/channel of data you constantly need to store, or $0.005/month. In what world is that too expensive or too much of a pain to deal with? If you're node is actually that busy, you're probably making that cost back per channel every minute, or better. > > RBF has a minimum incremental relay fee of 1sat/vByte by default. So if you > > use > > those 100 pre-signed transaction variants to do nothing more than sign every > > possible minimum incremental relay, you've covered a range of 1sat/vByte to > > 100sat/vByte. I believe that is sufficient to get mined for any block in > > Bitcoin's entire modern history. > > > > CPFP meanwhile requires two transactions, and thus extra bytes. Other than > > edge > > cases with very large transactions in low-fee environments, there's no > > circumstance where CPFP beats RBF. > > What I was referring to is that if we have the SIGHASH_SINGLE|ANYONECANPAY > we can combine many HTLC claims into one transaction, vs pre-signing means > we're stuck with a ton of individual transactions. Since SIGHASH_SINGLE requires one output per input, the savings you get by combining multiple SIGHASH_SINGLE transactions together aren't very significant. Just 18 bytes for nVersion, nLockTime, and the txin and txout size fields. The HTLC-timeout transaction is 166.5 vBytes, so that's a savings of just 11% Of course, if you _do_ need to fee bump and add an additional input, that input takes up space, and you'll probably need a change output. At which point you again would probably have been better off with a pre-signed transaction. You are also assuming there's lots of HTLC's in flight that need to be spent. That's very often not the case. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On Fri, Oct 20, 2023 at 09:03:49PM -0400, Matt Corallo wrote: > > What are anchor outputs used for other than increasing fees? > > > > Because if we've pre-signed the full fee range, there is simply no need for > > anchor outputs. Under any circumstance we can broadcast a transaction with a > > sufficiently high fee to get mined. > > > Indeed, that is what anchor outputs are for. Removing the pre-set feerate > solved a number of issues with edge-cases and helped address the > fee-inflation attack. Now, just using pre-signed transactions doesn't have > to re-introduce those issues - as long as the broadcaster gets to pick which > of the possible transactions they broadcast its just another transaction of > theirs. > > Still, I'm generally really dubious of the multiple pre-signed transaction > thing, (a) it would mean more fee overhead (not the end of the world for a > force-closure, but it sucks to have all these individual transactions > rolling around and be unable to batch), but more importantly (b) its a bunch > of overhead to keep track of a ton of variants across a sufficiently > granular set of feerates for it to not result in substantially overspending > on fees. Quite the contrary. Schnorr signatures are 64 bytes, so in situations like lightning where the transaction form is deterministically derived, signing 100 extra transactions requires just 6400 extra bytes. Even a very slow 100KB/s connection can transfer that in 64ms; latency will still dominate. RBF has a minimum incremental relay fee of 1sat/vByte by default. So if you use those 100 pre-signed transaction variants to do nothing more than sign every possible minimum incremental relay, you've covered a range of 1sat/vByte to 100sat/vByte. I believe that is sufficient to get mined for any block in Bitcoin's entire modern history. CPFP meanwhile requires two transactions, and thus extra bytes. Other than edge cases with very large transactions in low-fee environments, there's no circumstance where CPFP beats RBF. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On 10/20/23 8:15 PM, Peter Todd wrote: On Fri, Oct 20, 2023 at 05:05:48PM -0400, Matt Corallo wrote: Sadly this only is really viable for pre-anchor channels. With anchor channels the attack can be performed by either side of the closure, as the HTLCs are now, at max, only signed SIGHASH_SINGLE|ANYONECANPAY, allowing you to add more inputs and perform this attack even as the broadcaster. I don't think its really viable to walk that change back to fix this, as it also fixed plenty of other issues with channel usability and important edge-cases. What are anchor outputs used for other than increasing fees? Because if we've pre-signed the full fee range, there is simply no need for anchor outputs. Under any circumstance we can broadcast a transaction with a sufficiently high fee to get mined. Indeed, that is what anchor outputs are for. Removing the pre-set feerate solved a number of issues with edge-cases and helped address the fee-inflation attack. Now, just using pre-signed transactions doesn't have to re-introduce those issues - as long as the broadcaster gets to pick which of the possible transactions they broadcast its just another transaction of theirs. Still, I'm generally really dubious of the multiple pre-signed transaction thing, (a) it would mean more fee overhead (not the end of the world for a force-closure, but it sucks to have all these individual transactions rolling around and be unable to batch), but more importantly (b) its a bunch of overhead to keep track of a ton of variants across a sufficiently granular set of feerates for it to not result in substantially overspending on fees. Like I mentioned in the previous mail, this is really a policy bug - we're talking about a transaction pattern that might well happen where miners aren't getting the optimal value in transaction fees (potentially by a good bit). This needs to be fixed at the policy/Bitcoin Core layer, not in the lightning world (as much as its pretty resource-intensive to fix in the policy domain, I think). Matt ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
Could this be addressed with an OP_CSV_ALLINPUTS, a covenant opcode that requires *all* inputs to have a matching nSequence, and using `1 OP_CSV_ALLINPUTS` in the HTLC preimage branch? This would prevent using unconfirmed outputs in the HTLC-preimage-spending transaction entirely, which IIUC should protect it against the replacement cycling attack. (If desirable, this could alternatively be OP_CSV_OTHERINPUTS to allow the HTLC output itself to be spent immediately via the preimage branch, and only require that the other inputs added for fees are confirmed.) On Mon, Oct 16, 2023 at 8:03 PM Antoine Riard via bitcoin-dev < [email protected]> wrote: > (cross-posting mempool issues identified are exposing lightning chan to > loss of funds risks, other multi-party bitcoin apps might be affected) > > Hi, > > End of last year (December 2022), amid technical discussions on eltoo > payment channels and incentives compatibility of the mempool anti-DoS > rules, a new transaction-relay jamming attack affecting lightning channels > was discovered. > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning routing hops carrying HTLC traffic to loss of > funds security risks, both legacy and anchor output channels. A potential > exploitation plausibly happening even without network mempools congestion. > > Mitigations have been designed, implemented and deployed by all major > lightning implementations during the last months. > > Please find attached the release numbers, where the mitigations should be > present: > - LDK: v0.0.118 - CVE-2023 -40231 > - Eclair: v0.9.0 - CVE-2023-40232 > - LND: v.0.17.0-beta - CVE-2023-40233 > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > While neither replacement cycling attacks have been observed or reported > in the wild since the last ~10 months or experimented in real-world > conditions on bitcoin mainet, functional test is available exercising the > affected lightning channel against bitcoin core mempool (26.0 release > cycle). > > It is understood that a simple replacement cycling attack does not demand > privileged capabilities from an attacker (e.g no low-hashrate power) and > only access to basic bitcoin and lightning software. Yet I still think > executing such an attack successfully requests a fair amount of bitcoin > technical know-how and decent preparation. > > From my understanding of those issues, it is yet to be determined if the > mitigations deployed are robust enough in face of advanced replacement > cycling attackers, especially ones able to combine different classes of > transaction-relay jamming such as pinnings or vetted with more privileged > capabilities. > > Please find a list of potential affected bitcoin applications in this full > disclosure report using bitcoin script timelocks or multi-party > transactions, albeit no immediate security risk exposure as severe as the > ones affecting lightning has been identified. Only cursory review of > non-lightning applications has been conducted so far. > > There is a paper published summarizing replacement cycling attacks on the > lightning network: > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > ## Problem > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted > HTLC on incoming link and offered HTLC on outgoing link) should settle the > outgoing state with either a success or timeout before the incoming state > timelock becomes final and an asymmetric defavorable settlement might > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" > section 2.3 for a classical exposition of this lightning security property). > > Failure to satisfy this settlement requirement exposes a forwarding hop to > a loss of fund risk where the offered HTLC is spent by the outgoing link > counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming > link counterparty's HTLC-timeout. > > The specification mandates the incoming HTLC expiration timelock to be > spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC > expiration timelock, this exact interval value being an implementation and > node policy setting. As a minimal value, the specification recommends 34 > blocks of interval. If the timelock expiration I of the inbound HTLC is > equal to 100 from chain tip, the timelock expiration O of the outbound HTLC > must be equal to 66 blocks from chain tip, giving a reasonable buffer of > reaction to the lightning forwarding node. > > In the lack of cooperative off-chain settlement of the HTLC on the > outgoing link negotiated with the counterparty (either > `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the > lightning node should broadcast its commitment transaction. Once the > commitment is confirmed (if anchor and the 1 CSV encumbrance is present), > the lightning node broadcasts and confirms its HTLC-timeout before I height
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On 10/20/23 9:25 PM, Peter Todd wrote: On Fri, Oct 20, 2023 at 09:03:49PM -0400, Matt Corallo wrote: What are anchor outputs used for other than increasing fees? Because if we've pre-signed the full fee range, there is simply no need for anchor outputs. Under any circumstance we can broadcast a transaction with a sufficiently high fee to get mined. Indeed, that is what anchor outputs are for. Removing the pre-set feerate solved a number of issues with edge-cases and helped address the fee-inflation attack. Now, just using pre-signed transactions doesn't have to re-introduce those issues - as long as the broadcaster gets to pick which of the possible transactions they broadcast its just another transaction of theirs. Still, I'm generally really dubious of the multiple pre-signed transaction thing, (a) it would mean more fee overhead (not the end of the world for a force-closure, but it sucks to have all these individual transactions rolling around and be unable to batch), but more importantly (b) its a bunch of overhead to keep track of a ton of variants across a sufficiently granular set of feerates for it to not result in substantially overspending on fees. Quite the contrary. Schnorr signatures are 64 bytes, so in situations like lightning where the transaction form is deterministically derived, signing 100 extra transactions requires just 6400 extra bytes. Even a very slow 100KB/s connection can transfer that in 64ms; latency will still dominate. Lightning today isn't all that much data, but multiply it by 100 and we start racking up data enough that people may start to have to store a really material amount of data for larger nodes and dealing with that starts to be a much bigger pain then when we're talking about a GiB or twenty. RBF has a minimum incremental relay fee of 1sat/vByte by default. So if you use those 100 pre-signed transaction variants to do nothing more than sign every possible minimum incremental relay, you've covered a range of 1sat/vByte to 100sat/vByte. I believe that is sufficient to get mined for any block in Bitcoin's entire modern history. CPFP meanwhile requires two transactions, and thus extra bytes. Other than edge cases with very large transactions in low-fee environments, there's no circumstance where CPFP beats RBF. What I was referring to is that if we have the SIGHASH_SINGLE|ANYONECANPAY we can combine many HTLC claims into one transaction, vs pre-signing means we're stuck with a ton of individual transactions. Matt ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
> Let's say you have Alice, Bob and Caroll all "honest" routing hops > targeted by an attacker. They all have 3 independent 10 000 sats HTLC > in-flight on their outbound channels. > It is replaced by Mallory at T+2 with a HTLC-preimage X of 200 000 sats (+ > rbf penalty 1 sat / vb rule 4). Alice's HTLC-timeout is out of network > mempools. > Bob broadcasts her HTLC-timeout of 200 000 sats at T+3. It is replaced by > Mallory at T+4 with her HLTC-preimage Y of 200 000 sats (+ rbf penalty 1 > sat / vb rule 4 * 2). Bob's HTLC-timeout is out of network mempools. IIRC correctly, in your scenario, you're dealing with Carol -> Bob -> Alice. Mallory can only replace an HTLC-timeout transaction if she's directly connected with the peer she's targeting via a direct channel. She cannot unilaterally replace any transaction in the mempool solely with knowledge of the preimage. All HTLC transactions are two party contracts, with the public keys of both participants hard coded into the contract. A third party cannot unilaterally sweep an HTLC given only a preimage. I think it's worth taking a minute to step back here so we can establish some fundamental context. Once the timelock of an HTLC expires, and the receiver has direct knowledge of the preimage, a bidding war begins. If the receiver is able to confirm their success transaction in time, then they gain the funds, and the sender is able to pipeline the preimage backwards in the route, making all parties whole. If the sender is the one that wins out, then it's as if nothing happened, sans the fees paid at the last mile, all other hops are able to safely cancel their HTLC back on the chain. As mentioned elsewhere in the thread, most implementations today will watch the mempool for preimages, so they can resolve the incoming HTLC as quickly as possible off chain. The attack described critically relies on the ability of an attacker to pull off very precise transaction replacement globally across "the mempool". If any of the honest parties see the preimage, in the mempool, then the attack ends as they can settle back off chain, and if their timeout is able to confirm first, then the defender has actually gained funds. In addition, such an attack needs to be executed perfectly, for hours if not days. Within the LN, all nodes set a security parameter, the CLTV delta, that dictates how much time they have before the outgoing and incoming HTLCs expire. Increasing this value makes the attack more difficult, as they must maintain the superposition of: fees low enough to not get mined, but high enough to replace the defender's transaction, but not _too_ high, as then it'll be mined and everything is over. With each iteration, the attacker is also forced to increase the amount of fees they pay, increasing the likely hood that the transaction will be mined. If mined, the attack is moot. As mentioned in the OP, one thing the attacker can do is try and locate the defender's node to launch the replacement attack directly in their mempool. However, by replacing directly in the mempool of the defender, the defender will learn of the preimage! They can use that to just settle everything back, thereby completing the payment, and stopping the attack short. Even ignoring direct access to the defender's mempool, the attacker needs to somehow iteratively execute the replacement across a real network, with all the network jitter, propagation delay, and geographic heterogeneity that entails. If they're off by milliseconds, then either a confirmation occurs, or the preimage is revealed in the mempool. A canned local regtest env is one thing, the distributed system that is the Internet is another. Returning back to the bidding war: for anchor channels, the second level HTLCs allow the defender to attach arbitrary inputs for fee bumping purposes. Using this, they can iteratively increase their fee (via RBF) as the expiry deadline approaches. With each step, they further increase the cost for the attacker. On top of that, this attack cannot be launched indiscriminately across the network. Instead, it requires per-node set up by the attacker, as they need to consume UTXOs to create a chain of transactions they'll use to launch the replacement attack. These also need to be maintained in a similar state of non-confirming superposition. That's all to say that imo, this is a rather fragile attack, which requires: per-node setup, extremely precise timing and execution, non-confirming superposition of all transactions, and instant propagation across the entire network of the replacement transaction (somehow also being obscured from the PoV of the defender). The attack can be launched directly with a miner, or "into" their mempool, but that weakens the attack as if the miner broadcasts any of the preimage replacement transactions, then the defender can once again use that to extract the preimage and settle the incoming HTLC. -- Laolu ___
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On Fri, Oct 20, 2023 at 05:05:48PM -0400, Matt Corallo wrote: > Sadly this only is really viable for pre-anchor channels. With anchor > channels the attack can be performed by either side of the closure, as the > HTLCs are now, at max, only signed SIGHASH_SINGLE|ANYONECANPAY, allowing you > to add more inputs and perform this attack even as the broadcaster. > > I don't think its really viable to walk that change back to fix this, as it > also fixed plenty of other issues with channel usability and important > edge-cases. What are anchor outputs used for other than increasing fees? Because if we've pre-signed the full fee range, there is simply no need for anchor outputs. Under any circumstance we can broadcast a transaction with a sufficiently high fee to get mined. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
I think if we apply this presigned fee multiplier idea to HTLC spends, we can prevent replacement cycles from happening. We could modify HTLC scripts so that *both* parties can only spend the HTLC via presigned second-stage transactions, and we can always sign those with SIGHASH_ALL. This will prevent the attacker from adding inputs to their presigned transaction, so (AFAICT) a replacement cycling attack becomes impossible. The tradeoff is more bookkeeping and less fee granularity when claiming HTLCs on chain. On Fri, Oct 20, 2023 at 11:04 AM Peter Todd via bitcoin-dev wrote: > > On Fri, Oct 20, 2023 at 10:31:03AM +, Peter Todd via bitcoin-dev wrote: > > As I have suggested before, the correct way to do pre-signed transactions > > is to > > pre-sign enough *different* transactions to cover all reasonable needs for > > bumping fees. Even if you just increase the fee by 2x each time, > > pre-signing 10 > > different replacement transactions covers a fee range of 1024x. And you > > obviously can improve on this by increasing the multiplier towards the end > > of > > the range. > > To be clear, when I say "increasing the multiplier", I mean, starting with a > smaller multiplier at the beginning of the range, and ending with a bigger > one. > > Eg feebumping with fee increases pre-signed for something like: > > 1.1 > 1.2 > 1.4 > 1.8 > 2.6 > 4.2 > 7.4 > > etc. > > That would use most of the range for smaller bumps, as a %, with larger % > bumps > reserved for the end where our strategy is changing to something more > "scorched-earth" > > And of course, applying this idea properly to commitment transactions will > mean > that the replacements may have HTLCs removed, when their value drops below the > fees necessary to get those outputs mined. > > Note too that we can sign simultaneous variants of transactions that deduct > the > fees from different party's outputs. Eg Alice can give Bob the ability to > broadcast higher and higher fee txs, taking the fees from Bob's output(s), and > Bob can give Alice the same ability, taking the fees from Alice's output(s). I > haven't thought through how this would work with musig. But you can certainly > do that with plain old OP_CheckMultisig. > > -- > https://petertodd.org 'peter'[:-1]@petertodd.org > ___ > bitcoin-dev mailing list > [email protected] > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
Sadly this only is really viable for pre-anchor channels. With anchor channels the attack can be performed by either side of the closure, as the HTLCs are now, at max, only signed SIGHASH_SINGLE|ANYONECANPAY, allowing you to add more inputs and perform this attack even as the broadcaster. I don't think its really viable to walk that change back to fix this, as it also fixed plenty of other issues with channel usability and important edge-cases. I'll highlight that fixing this issue on the lightning end isn't really the right approach generally - we're talking about a case where a lightning counterparty engineered a transaction broadcast ordering such that miners are *not* including the optimal set of transactions for fee revenue. Given such a scenario exists (and its not unrealistic to think someone might wish to engineer such a situation), the fix ultimately needs to lie with Bitcoin Core (or other parts of the mining stack). Now, fixing this in the Bitcoin Core stack is no trivial deal - the reason for this attack is to keep enough history to fix it Bitcoin Core would need unbounded memory. However, its not hard to imagine a simple external piece of software which monitors the mempool for transactions which were replaced out but which may be able to re-enter the mempool later with other replacements and store them on disk. From there, this software could optimize the revenue of block template selection, while also accidentally fixing this issue. Matt On 10/20/23 2:35 PM, Matt Morehouse via bitcoin-dev wrote: I think if we apply this presigned fee multiplier idea to HTLC spends, we can prevent replacement cycles from happening. We could modify HTLC scripts so that *both* parties can only spend the HTLC via presigned second-stage transactions, and we can always sign those with SIGHASH_ALL. This will prevent the attacker from adding inputs to their presigned transaction, so (AFAICT) a replacement cycling attack becomes impossible. The tradeoff is more bookkeeping and less fee granularity when claiming HTLCs on chain. On Fri, Oct 20, 2023 at 11:04 AM Peter Todd via bitcoin-dev wrote: On Fri, Oct 20, 2023 at 10:31:03AM +, Peter Todd via bitcoin-dev wrote: As I have suggested before, the correct way to do pre-signed transactions is to pre-sign enough *different* transactions to cover all reasonable needs for bumping fees. Even if you just increase the fee by 2x each time, pre-signing 10 different replacement transactions covers a fee range of 1024x. And you obviously can improve on this by increasing the multiplier towards the end of the range. To be clear, when I say "increasing the multiplier", I mean, starting with a smaller multiplier at the beginning of the range, and ending with a bigger one. Eg feebumping with fee increases pre-signed for something like: 1.1 1.2 1.4 1.8 2.6 4.2 7.4 etc. That would use most of the range for smaller bumps, as a %, with larger % bumps reserved for the end where our strategy is changing to something more "scorched-earth" And of course, applying this idea properly to commitment transactions will mean that the replacements may have HTLCs removed, when their value drops below the fees necessary to get those outputs mined. Note too that we can sign simultaneous variants of transactions that deduct the fees from different party's outputs. Eg Alice can give Bob the ability to broadcast higher and higher fee txs, taking the fees from Bob's output(s), and Bob can give Alice the same ability, taking the fees from Alice's output(s). I haven't thought through how this would work with musig. But you can certainly do that with plain old OP_CheckMultisig. -- https://petertodd.org 'peter'[:-1]@petertodd.org ___ bitcoin-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev ___ bitcoin-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On Fri, Oct 20, 2023 at 10:31:03AM +, Peter Todd via bitcoin-dev wrote: > As I have suggested before, the correct way to do pre-signed transactions is > to > pre-sign enough *different* transactions to cover all reasonable needs for > bumping fees. Even if you just increase the fee by 2x each time, pre-signing > 10 > different replacement transactions covers a fee range of 1024x. And you > obviously can improve on this by increasing the multiplier towards the end of > the range. To be clear, when I say "increasing the multiplier", I mean, starting with a smaller multiplier at the beginning of the range, and ending with a bigger one. Eg feebumping with fee increases pre-signed for something like: 1.1 1.2 1.4 1.8 2.6 4.2 7.4 etc. That would use most of the range for smaller bumps, as a %, with larger % bumps reserved for the end where our strategy is changing to something more "scorched-earth" And of course, applying this idea properly to commitment transactions will mean that the replacements may have HTLCs removed, when their value drops below the fees necessary to get those outputs mined. Note too that we can sign simultaneous variants of transactions that deduct the fees from different party's outputs. Eg Alice can give Bob the ability to broadcast higher and higher fee txs, taking the fees from Bob's output(s), and Bob can give Alice the same ability, taking the fees from Alice's output(s). I haven't thought through how this would work with musig. But you can certainly do that with plain old OP_CheckMultisig. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On Tue, Oct 17, 2023 at 10:34:04AM +, ZmnSCPxj via bitcoin-dev wrote: > Good morning Antoine et al., > > Let me try to rephrase the core of the attack. > > There exists these nodes on the LN (letters `A`, `B`, and `C` are nodes, `==` > are channels): > > A = B = C > > `A` routes `A->B->C`. > > The timelocks, for example, could be: > >A->B timeelock = 144 >B->C timelock = 100 > > The above satisfies the LN BOLT requirements, as long as `B` has a > `cltv_expiry_delta` of 44 or lower. > > After `B` forwards the HTLC `B->C`, C suddenly goes offline, and all the > signed transactions --- commitment transaction and HTLC-timeout transactions > --- are "stuck" at the feerate at the time. > > At block height 100, `B` notices the `B->C` HTLC timelock is expired without > `C` having claimed it, so `B` forces the `BC` channel onchain. > However, onchain feerates have risen and the commitment transaction and > HTLC-timeout transaction do not confirm. The problem here is we're failing to use RBF. As I have suggested before, the correct way to do pre-signed transactions is to pre-sign enough *different* transactions to cover all reasonable needs for bumping fees. Even if you just increase the fee by 2x each time, pre-signing 10 different replacement transactions covers a fee range of 1024x. And you obviously can improve on this by increasing the multiplier towards the end of the range. Increasing per-tx (temporary) storage and bandwidth costs by ~10x or even ~100x is not a big deal in the context of a highly scalable protocol like Lightning. There is zero reason why the B->C transactions should be getting stuck. This is a major failing of the Lightning protocol that should be fixed. And of course, this fix should be applied to other aspects of the lightning protocol, such as channel opens, etc. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ Lightning-dev mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
That certainly helps, yes, and I think many nodes do something akin to this already, but I'm not sure we can say that the problem has been fixed if the victim has to spend way more than the prevailing mempool fees (and potentially burn a large % of their HTLC value) :). Matt On 10/19/23 12:23 PM, Matt Morehouse wrote: On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev wrote: There appears to be some confusion about this issue and the mitigations. To be clear, the deployed mitigations are not expected to fix this issue, its arguable if they provide anything more than a PR statement. There are two discussed mitigations here - mempool scanning and transaction re-signing/re-broadcasting. Mempool scanning relies on regularly checking the mempool of a local node to see if we can catch the replacement cycle mid-cycle. It only works if wee see the first transaction before the second transaction replaces it. Today, a large majority of lightning nodes run on machines with a Bitcoin node on the same IP address, making it very clear what the "local node" of the lightning node is. An attacker can trivially use this information to connect to said local node and do the replacement quickly, preventing the victim from seeing the replacement. More generally, however, similar discoverability is true for mining pools. An attacker performing this attack is likely to do the replacement attack on a miner's node directly, potentially reducing the reach of the intermediate transaction to only miners, such that the victim can never discover it at all. The second mitigation is similarly pathetic. Re-signing and re-broadcasting the victim's transaction in an attempt to get it to miners even if its been removed may work, if the attacker is super lazy and didn't finish writing their attack system. If the attacker is connected to a large majority of hashrate (which has historically been fairly doable), they can simply do their replacement in a cycle aggressively and arbitrarily reduce the probability that the victim's transaction gets confirmed. What if the honest node aggressively fee-bumps and retransmits the HTLC-timeout as the CLTV delta deadline approaches, as suggested by Ziggie? Say, within 10 blocks of the deadline, the honest node starts increasing the fee by 1/10th the HTLC value for each non-confirmation. This "scorched earth" approach may cost the honest node considerable fees, but it will cost the attacker even more, since each attacker replacement needs to burn at least as much as the HTLC-timeout fees, and the attacker will need to do a replacement every time the honest node fee bumps. I think this fee-bumping policy will provide sufficient defense even if the attacker is replacement-cycling directly in miners' mempools and the victim has no visibility into the attack. Now, the above is all true in a spherical cow kinda world, and the P2P network has plenty of slow nodes and strange behavior. Its possible that these mitigations might, by some stroke of luck, happen to catch such an attack and prevent it, because something took longer than the attacker intended or whatever. But, that's a far cry from any kind of material "fix" for the issue. Ultimately the only fix for this issue will be when miners keep a history of transactions they've seen and try them again after they may be able to enter the mempool because of an attack like this. Matt On 10/16/23 12:57 PM, Antoine Riard wrote: (cross-posting mempool issues identified are exposing lightning chan to loss of funds risks, other multi-party bitcoin apps might be affected) Hi, End of last year (December 2022), amid technical discussions on eltoo payment channels and incentives compatibility of the mempool anti-DoS rules, a new transaction-relay jamming attack affecting lightning channels was discovered. After careful analysis, it turns out this attack is practical and immediately exposed lightning routing hops carrying HTLC traffic to loss of funds security risks, both legacy and anchor output channels. A potential exploitation plausibly happening even without network mempools congestion. Mitigations have been designed, implemented and deployed by all major lightning implementations during the last months. Please find attached the release numbers, where the mitigations should be present: - LDK: v0.0.118 - CVE-2023 -40231 - Eclair: v0.9.0 - CVE-2023-40232 - LND: v.0.17.0-beta - CVE-2023-40233 - Core-Lightning: v.23.08.01 - CVE-2023-40234 While neither replacement cycling attacks have been observed or reported in the wild since the last ~10 months or experimented in real-world conditions on bitcoin mainet, functional test is available exercising the affected lightning channel against bitcoin core mempool (26.0 release cycle). It is understood that a simple replacement cycling attack does not demand privileged capabilities from an attacker (e.g no low-hashrate power) and only access to basic
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
> As the CLTV > delta deadline approaches, the fees in case 2 may be 50%, 80%, even > 100% of the HTLC value under such a scorched earth policy. A replacement-cycling attacker can afford to pay 100% of the HTLC value under the defender scorched earth policy and still realize an economic gain. Let's say you have Alice, Bob and Caroll all "honest" routing hops targeted by an attacker. They all have 3 independent 10 000 sats HTLC in-flight on their outbound channels. Under this defensive fee scorched earth policy, Alice broadcasts her HTLC-timeout at T + 1 with 10 000 sats committed as absolute fee. It is replaced by Mallory at T+2 with a HTLC-preimage X of 200 000 sats (+ rbf penalty 1 sat / vb rule 4). Alice's HTLC-timeout is out of network mempools. Bob broadcasts her HTLC-timeout of 200 000 sats at T+3. It is replaced by Mallory at T+4 with her HLTC-preimage Y of 200 000 sats (+ rbf penalty 1 sat / vb rule 4 * 2). Bob's HTLC-timeout is out of network mempools. HTLC-preimage Y conflicts with HTLC-preimage X too (here comes the multiplied by 2 of the rbf penalty). Caroll broadcasts her HTLC-timeout of 200 000 sats at T+5. It is replaced by Mallory at T+6 with her HTLC-preimage Z of 200 000 sats (+rbf penalty 1 sat / vb rule 4 * 3). Caroll's HTLC-timeout is out of network mempools. HTLC-preimage Z conflicts with HTLC-preimage Z too (here comes the multiplied by 3 of rbf penalty). At any time if Mallory's HTLC-preimage enters into the top mempool feerates group (due to the accumulated rbf penalty), one unconfirmed ancestor can be double-spent to evict out the HTLC-preimage. If Mallory succeeds the replacement cycling, she might be at loss of 10 000 sats + rbf penalty cost for each rebroadcast attempt of the victim. However she wins Alice + Bob + Carol HTLC value of 200 000 sats each. Assuming 5 rebroadcasts per block (even on random timers) multiplied by 3 victims, 200 bytes of HTLC-preimage and cltv_delta of 144 blocks, the total attacker cost is 432 000 sats. The economic gain realized is 168 000 sats. Sounds each additional victim has a cost of 144 000 sats, whatever the HTLC value targeted. Thanks for checking the fees math and replacement rules, though it sounds correct to me. (And here without introducing more favorable assumptions to the attacker, like mempool spikes where the "honest" HTLC-timeout transactions can be let floating in network mempools). Best, Antoine Le jeu. 19 oct. 2023 à 18:54, Matt Morehouse a écrit : > On Thu, Oct 19, 2023 at 5:22 PM Antoine Riard > wrote: > > > > Hi Matt, > > > > This mitigation is mentioned in the attached paper (see subsection 3.4 > defensive fee-rebroadcasting) > > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > > > As soon as you start to have a bit of a mempool backlog and the > defensive fractional fee HTLC-timeout stays stuck, it gives the advantage > to the attacker again. > > > > Beyond that, I think an attacker can replace-cycle multiple honest > HTLC-timeout with a single malicious HTLC-preimage (with a sequence of > replacement, not concurrently) paying the absolute fee, while only > encumbering the RBF penalty. I didn't test this specific behavior, though > the "fees" math doesn't seem at the advantage of the defenders at first > sight. > > Right, some replacement cycles can be avoided by the attacker to > reduce the cost of the attack. But with the defender implementing a > scorched-earth fee bumping policy, eventually either (1) the > HTLC-timeout *will* confirm in the next block or (2) the attacker must > pay more fees than the HTLC-timeout fees to replace it. As the CLTV > delta deadline approaches, the fees in case 2 may be 50%, 80%, even > 100% of the HTLC value under such a scorched earth policy. So even if > the attacker only has to do one replacement cycle right before the > deadline, the attack can be made unprofitable. And in practice, with > HTLC values significantly greater than the next-block fee cost, the > attacker will need to do multiple replacements as the deadline > approaches. > > And of course this linear scorched earth policy is just an > illustration. We should further tune the fee bumping curve across the > full CLTV delta to ensure minimal fees paid when not under attack. > But as we approach the deadline, it makes sense to become very > aggressive both to get our transaction confirmed during high mempool > congestion and to punish replacement-cycling attackers. > > > > > Best, > > Antoine > > > > Le jeu. 19 oct. 2023 à 17:23, Matt Morehouse > a écrit : > >> > >> On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev > >> wrote: > >> > > >> > There appears to be some confusion about this issue and the > mitigations. To be clear, the deployed > >> > mitigations are not expected to fix this issue, its arguable if they > provide anything more than a PR > >> > statement. > >> > > >> > There are two discussed mitigations here - mempool scanning
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On Thu, Oct 19, 2023 at 5:22 PM Antoine Riard wrote: > > Hi Matt, > > This mitigation is mentioned in the attached paper (see subsection 3.4 > defensive fee-rebroadcasting) > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > As soon as you start to have a bit of a mempool backlog and the defensive > fractional fee HTLC-timeout stays stuck, it gives the advantage to the > attacker again. > > Beyond that, I think an attacker can replace-cycle multiple honest > HTLC-timeout with a single malicious HTLC-preimage (with a sequence of > replacement, not concurrently) paying the absolute fee, while only > encumbering the RBF penalty. I didn't test this specific behavior, though the > "fees" math doesn't seem at the advantage of the defenders at first sight. Right, some replacement cycles can be avoided by the attacker to reduce the cost of the attack. But with the defender implementing a scorched-earth fee bumping policy, eventually either (1) the HTLC-timeout *will* confirm in the next block or (2) the attacker must pay more fees than the HTLC-timeout fees to replace it. As the CLTV delta deadline approaches, the fees in case 2 may be 50%, 80%, even 100% of the HTLC value under such a scorched earth policy. So even if the attacker only has to do one replacement cycle right before the deadline, the attack can be made unprofitable. And in practice, with HTLC values significantly greater than the next-block fee cost, the attacker will need to do multiple replacements as the deadline approaches. And of course this linear scorched earth policy is just an illustration. We should further tune the fee bumping curve across the full CLTV delta to ensure minimal fees paid when not under attack. But as we approach the deadline, it makes sense to become very aggressive both to get our transaction confirmed during high mempool congestion and to punish replacement-cycling attackers. > > Best, > Antoine > > Le jeu. 19 oct. 2023 à 17:23, Matt Morehouse a > écrit : >> >> On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev >> wrote: >> > >> > There appears to be some confusion about this issue and the mitigations. >> > To be clear, the deployed >> > mitigations are not expected to fix this issue, its arguable if they >> > provide anything more than a PR >> > statement. >> > >> > There are two discussed mitigations here - mempool scanning and >> > transaction re-signing/re-broadcasting. >> > >> > Mempool scanning relies on regularly checking the mempool of a local node >> > to see if we can catch the >> > replacement cycle mid-cycle. It only works if wee see the first >> > transaction before the second >> > transaction replaces it. >> > >> > Today, a large majority of lightning nodes run on machines with a Bitcoin >> > node on the same IP >> > address, making it very clear what the "local node" of the lightning node >> > is. An attacker can >> > trivially use this information to connect to said local node and do the >> > replacement quickly, >> > preventing the victim from seeing the replacement. >> > >> > More generally, however, similar discoverability is true for mining pools. >> > An attacker performing >> > this attack is likely to do the replacement attack on a miner's node >> > directly, potentially reducing >> > the reach of the intermediate transaction to only miners, such that the >> > victim can never discover it >> > at all. >> > >> > The second mitigation is similarly pathetic. Re-signing and >> > re-broadcasting the victim's transaction >> > in an attempt to get it to miners even if its been removed may work, if >> > the attacker is super lazy >> > and didn't finish writing their attack system. If the attacker is >> > connected to a large majority of >> > hashrate (which has historically been fairly doable), they can simply do >> > their replacement in a >> > cycle aggressively and arbitrarily reduce the probability that the >> > victim's transaction gets confirmed. >> >> What if the honest node aggressively fee-bumps and retransmits the >> HTLC-timeout as the CLTV delta deadline approaches, as suggested by >> Ziggie? Say, within 10 blocks of the deadline, the honest node starts >> increasing the fee by 1/10th the HTLC value for each non-confirmation. >> >> This "scorched earth" approach may cost the honest node considerable >> fees, but it will cost the attacker even more, since each attacker >> replacement needs to burn at least as much as the HTLC-timeout fees, >> and the attacker will need to do a replacement every time the honest >> node fee bumps. >> >> I think this fee-bumping policy will provide sufficient defense even >> if the attacker is replacement-cycling directly in miners' mempools >> and the victim has no visibility into the attack. >> >> > >> > Now, the above is all true in a spherical cow kinda world, and the P2P >> > network has plenty of slow >> > nodes and strange behavior. Its possible that these
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
Hi Matt, This mitigation is mentioned in the attached paper (see subsection 3.4 defensive fee-rebroadcasting) https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf As soon as you start to have a bit of a mempool backlog and the defensive fractional fee HTLC-timeout stays stuck, it gives the advantage to the attacker again. Beyond that, I think an attacker can replace-cycle multiple honest HTLC-timeout with a single malicious HTLC-preimage (with a sequence of replacement, not concurrently) paying the absolute fee, while only encumbering the RBF penalty. I didn't test this specific behavior, though the "fees" math doesn't seem at the advantage of the defenders at first sight. Best, Antoine Le jeu. 19 oct. 2023 à 17:23, Matt Morehouse a écrit : > On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev > wrote: > > > > There appears to be some confusion about this issue and the mitigations. > To be clear, the deployed > > mitigations are not expected to fix this issue, its arguable if they > provide anything more than a PR > > statement. > > > > There are two discussed mitigations here - mempool scanning and > transaction re-signing/re-broadcasting. > > > > Mempool scanning relies on regularly checking the mempool of a local > node to see if we can catch the > > replacement cycle mid-cycle. It only works if wee see the first > transaction before the second > > transaction replaces it. > > > > Today, a large majority of lightning nodes run on machines with a > Bitcoin node on the same IP > > address, making it very clear what the "local node" of the lightning > node is. An attacker can > > trivially use this information to connect to said local node and do the > replacement quickly, > > preventing the victim from seeing the replacement. > > > > More generally, however, similar discoverability is true for mining > pools. An attacker performing > > this attack is likely to do the replacement attack on a miner's node > directly, potentially reducing > > the reach of the intermediate transaction to only miners, such that the > victim can never discover it > > at all. > > > > The second mitigation is similarly pathetic. Re-signing and > re-broadcasting the victim's transaction > > in an attempt to get it to miners even if its been removed may work, if > the attacker is super lazy > > and didn't finish writing their attack system. If the attacker is > connected to a large majority of > > hashrate (which has historically been fairly doable), they can simply do > their replacement in a > > cycle aggressively and arbitrarily reduce the probability that the > victim's transaction gets confirmed. > > What if the honest node aggressively fee-bumps and retransmits the > HTLC-timeout as the CLTV delta deadline approaches, as suggested by > Ziggie? Say, within 10 blocks of the deadline, the honest node starts > increasing the fee by 1/10th the HTLC value for each non-confirmation. > > This "scorched earth" approach may cost the honest node considerable > fees, but it will cost the attacker even more, since each attacker > replacement needs to burn at least as much as the HTLC-timeout fees, > and the attacker will need to do a replacement every time the honest > node fee bumps. > > I think this fee-bumping policy will provide sufficient defense even > if the attacker is replacement-cycling directly in miners' mempools > and the victim has no visibility into the attack. > > > > > Now, the above is all true in a spherical cow kinda world, and the P2P > network has plenty of slow > > nodes and strange behavior. Its possible that these mitigations might, > by some stroke of luck, > > happen to catch such an attack and prevent it, because something took > longer than the attacker > > intended or whatever. But, that's a far cry from any kind of material > "fix" for the issue. > > > > Ultimately the only fix for this issue will be when miners keep a > history of transactions they've > > seen and try them again after they may be able to enter the mempool > because of an attack like this. > > > > Matt > > > > On 10/16/23 12:57 PM, Antoine Riard wrote: > > > (cross-posting mempool issues identified are exposing lightning chan > to loss of funds risks, other > > > multi-party bitcoin apps might be affected) > > > > > > Hi, > > > > > > End of last year (December 2022), amid technical discussions on eltoo > payment channels and > > > incentives compatibility of the mempool anti-DoS rules, a new > transaction-relay jamming attack > > > affecting lightning channels was discovered. > > > > > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning > > > routing hops carrying HTLC traffic to loss of funds security risks, > both legacy and anchor output > > > channels. A potential exploitation plausibly happening even without > network mempools congestion. > > > > > > Mitigations have been designed, implemented and deployed by all major >
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev wrote: > > There appears to be some confusion about this issue and the mitigations. To > be clear, the deployed > mitigations are not expected to fix this issue, its arguable if they provide > anything more than a PR > statement. > > There are two discussed mitigations here - mempool scanning and transaction > re-signing/re-broadcasting. > > Mempool scanning relies on regularly checking the mempool of a local node to > see if we can catch the > replacement cycle mid-cycle. It only works if wee see the first transaction > before the second > transaction replaces it. > > Today, a large majority of lightning nodes run on machines with a Bitcoin > node on the same IP > address, making it very clear what the "local node" of the lightning node is. > An attacker can > trivially use this information to connect to said local node and do the > replacement quickly, > preventing the victim from seeing the replacement. > > More generally, however, similar discoverability is true for mining pools. An > attacker performing > this attack is likely to do the replacement attack on a miner's node > directly, potentially reducing > the reach of the intermediate transaction to only miners, such that the > victim can never discover it > at all. > > The second mitigation is similarly pathetic. Re-signing and re-broadcasting > the victim's transaction > in an attempt to get it to miners even if its been removed may work, if the > attacker is super lazy > and didn't finish writing their attack system. If the attacker is connected > to a large majority of > hashrate (which has historically been fairly doable), they can simply do > their replacement in a > cycle aggressively and arbitrarily reduce the probability that the victim's > transaction gets confirmed. What if the honest node aggressively fee-bumps and retransmits the HTLC-timeout as the CLTV delta deadline approaches, as suggested by Ziggie? Say, within 10 blocks of the deadline, the honest node starts increasing the fee by 1/10th the HTLC value for each non-confirmation. This "scorched earth" approach may cost the honest node considerable fees, but it will cost the attacker even more, since each attacker replacement needs to burn at least as much as the HTLC-timeout fees, and the attacker will need to do a replacement every time the honest node fee bumps. I think this fee-bumping policy will provide sufficient defense even if the attacker is replacement-cycling directly in miners' mempools and the victim has no visibility into the attack. > > Now, the above is all true in a spherical cow kinda world, and the P2P > network has plenty of slow > nodes and strange behavior. Its possible that these mitigations might, by > some stroke of luck, > happen to catch such an attack and prevent it, because something took longer > than the attacker > intended or whatever. But, that's a far cry from any kind of material "fix" > for the issue. > > Ultimately the only fix for this issue will be when miners keep a history of > transactions they've > seen and try them again after they may be able to enter the mempool because > of an attack like this. > > Matt > > On 10/16/23 12:57 PM, Antoine Riard wrote: > > (cross-posting mempool issues identified are exposing lightning chan to > > loss of funds risks, other > > multi-party bitcoin apps might be affected) > > > > Hi, > > > > End of last year (December 2022), amid technical discussions on eltoo > > payment channels and > > incentives compatibility of the mempool anti-DoS rules, a new > > transaction-relay jamming attack > > affecting lightning channels was discovered. > > > > After careful analysis, it turns out this attack is practical and > > immediately exposed lightning > > routing hops carrying HTLC traffic to loss of funds security risks, both > > legacy and anchor output > > channels. A potential exploitation plausibly happening even without network > > mempools congestion. > > > > Mitigations have been designed, implemented and deployed by all major > > lightning implementations > > during the last months. > > > > Please find attached the release numbers, where the mitigations should be > > present: > > - LDK: v0.0.118 - CVE-2023 -40231 > > - Eclair: v0.9.0 - CVE-2023-40232 > > - LND: v.0.17.0-beta - CVE-2023-40233 > > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > > > While neither replacement cycling attacks have been observed or reported in > > the wild since the last > > ~10 months or experimented in real-world conditions on bitcoin mainet, > > functional test is available > > exercising the affected lightning channel against bitcoin core mempool > > (26.0 release cycle). > > > > It is understood that a simple replacement cycling attack does not demand > > privileged capabilities > > from an attacker (e.g no low-hashrate power) and only access to basic > > bitcoin and lightning > > software. Yet I still think executing such
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
Thanks Antoine for your work on this issue. I confirm that eclair v0.9.0 contains the migitations described. Eclair has been watching the mempool for preimages since its very early versions (years ago), relying on Bitcoin Core's ZMQ notifications for incoming transactions. I believe this guarantees that we see the HTLC success transaction, even if it is immediately replaced (as long as we don't overflow the ZMQ limits). I agree with Matt though that more fundamental work most likely needs to happen at the bitcoin layer to allow L2 protocols to be more robust against that class of attacks. Thanks, Bastien Le mer. 18 oct. 2023 à 11:07, Antoine Riard via bitcoin-dev < [email protected]> a écrit : > The disclosure mails noted a 3rd mitigation beyond mempool scanning and > transaction re-signing / re-broadcasting, namely bumping CLTV delta. > > Generally bumping CLTV delta is a basic line of mitigations for a lot of > lightning attacks, as it gives opportunity to node operators to intervene > and re-broadcast their time-sensitive transactions on other interfaces (e.g > a secondary full-node if the first one is eclipsed). > > About the second mitigation transaction re-signing, if done correctly at > least sounds to put an economic cost (denominated in fees / feerates) on > the attack. This is unclear to me if the game-theory of this cost holds. > > One thing which sounds to me making the attack harder is stratum v2 > deployment, as you're increasing the number of miners which might do their > own block templates, and therefore the number of miners' mempools where an > attacker has to successfully continuously replace in cycles channels > counterparties transactions. > > A replacement buffer or history of transactions at the mempool level might > be a mitigation to this attack. I believe this is yet to be seen if it can > be made robust enough. > > I don't know if folks like tadge or rusty who have been involved in the > early design of lightning have more ideas of mitigations. Fees was noted as > a hard issue in the original paper. > > Le mer. 18 oct. 2023 à 01:17, Matt Corallo a > écrit : > >> There appears to be some confusion about this issue and the mitigations. >> To be clear, the deployed >> mitigations are not expected to fix this issue, its arguable if they >> provide anything more than a PR >> statement. >> >> There are two discussed mitigations here - mempool scanning and >> transaction re-signing/re-broadcasting. >> >> Mempool scanning relies on regularly checking the mempool of a local node >> to see if we can catch the >> replacement cycle mid-cycle. It only works if wee see the first >> transaction before the second >> transaction replaces it. >> >> Today, a large majority of lightning nodes run on machines with a Bitcoin >> node on the same IP >> address, making it very clear what the "local node" of the lightning node >> is. An attacker can >> trivially use this information to connect to said local node and do the >> replacement quickly, >> preventing the victim from seeing the replacement. >> >> More generally, however, similar discoverability is true for mining >> pools. An attacker performing >> this attack is likely to do the replacement attack on a miner's node >> directly, potentially reducing >> the reach of the intermediate transaction to only miners, such that the >> victim can never discover it >> at all. >> >> The second mitigation is similarly pathetic. Re-signing and >> re-broadcasting the victim's transaction >> in an attempt to get it to miners even if its been removed may work, if >> the attacker is super lazy >> and didn't finish writing their attack system. If the attacker is >> connected to a large majority of >> hashrate (which has historically been fairly doable), they can simply do >> their replacement in a >> cycle aggressively and arbitrarily reduce the probability that the >> victim's transaction gets confirmed. >> >> Now, the above is all true in a spherical cow kinda world, and the P2P >> network has plenty of slow >> nodes and strange behavior. Its possible that these mitigations might, by >> some stroke of luck, >> happen to catch such an attack and prevent it, because something took >> longer than the attacker >> intended or whatever. But, that's a far cry from any kind of material >> "fix" for the issue. >> >> Ultimately the only fix for this issue will be when miners keep a history >> of transactions they've >> seen and try them again after they may be able to enter the mempool >> because of an attack like this. >> >> Matt >> >> On 10/16/23 12:57 PM, Antoine Riard wrote: >> > (cross-posting mempool issues identified are exposing lightning chan to >> loss of funds risks, other >> > multi-party bitcoin apps might be affected) >> > >> > Hi, >> > >> > End of last year (December 2022), amid technical discussions on eltoo >> payment channels and >> > incentives compatibility of the mempool anti-DoS rules, a new >> transaction-relay jamming
Re: [Lightning-dev] [bitcoin-dev] Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
Hi Antoine, Thanks for this great write up, and also your diligence in reporting this issue to the various implementations, and game planning with us re mitigations and attack scenarios. One small clarification: all of lnd's relevant mitigations were in place by lnd v0.16.1-beta [1], which was released on April 24th 2023. Since then we've fixed some performance regressions introduced due to some of the mitigations (mempool watching), and in 0.17.1 we'll start to use the new `gettxspendingprevout` RPC call with bitcoind to further reduce load. [1]: https://github.com/lightningnetwork/lnd/releases/tag/v0.16.1-beta -- Laolu On Mon, Oct 16, 2023 at 10:04 AM Antoine Riard via bitcoin-dev < [email protected]> wrote: > (cross-posting mempool issues identified are exposing lightning chan to > loss of funds risks, other multi-party bitcoin apps might be affected) > > Hi, > > End of last year (December 2022), amid technical discussions on eltoo > payment channels and incentives compatibility of the mempool anti-DoS > rules, a new transaction-relay jamming attack affecting lightning channels > was discovered. > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning routing hops carrying HTLC traffic to loss of > funds security risks, both legacy and anchor output channels. A potential > exploitation plausibly happening even without network mempools congestion. > > Mitigations have been designed, implemented and deployed by all major > lightning implementations during the last months. > > Please find attached the release numbers, where the mitigations should be > present: > - LDK: v0.0.118 - CVE-2023 -40231 > - Eclair: v0.9.0 - CVE-2023-40232 > - LND: v.0.17.0-beta - CVE-2023-40233 > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > While neither replacement cycling attacks have been observed or reported > in the wild since the last ~10 months or experimented in real-world > conditions on bitcoin mainet, functional test is available exercising the > affected lightning channel against bitcoin core mempool (26.0 release > cycle). > > It is understood that a simple replacement cycling attack does not demand > privileged capabilities from an attacker (e.g no low-hashrate power) and > only access to basic bitcoin and lightning software. Yet I still think > executing such an attack successfully requests a fair amount of bitcoin > technical know-how and decent preparation. > > From my understanding of those issues, it is yet to be determined if the > mitigations deployed are robust enough in face of advanced replacement > cycling attackers, especially ones able to combine different classes of > transaction-relay jamming such as pinnings or vetted with more privileged > capabilities. > > Please find a list of potential affected bitcoin applications in this full > disclosure report using bitcoin script timelocks or multi-party > transactions, albeit no immediate security risk exposure as severe as the > ones affecting lightning has been identified. Only cursory review of > non-lightning applications has been conducted so far. > > There is a paper published summarizing replacement cycling attacks on the > lightning network: > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > ## Problem > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted > HTLC on incoming link and offered HTLC on outgoing link) should settle the > outgoing state with either a success or timeout before the incoming state > timelock becomes final and an asymmetric defavorable settlement might > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" > section 2.3 for a classical exposition of this lightning security property). > > Failure to satisfy this settlement requirement exposes a forwarding hop to > a loss of fund risk where the offered HTLC is spent by the outgoing link > counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming > link counterparty's HTLC-timeout. > > The specification mandates the incoming HTLC expiration timelock to be > spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC > expiration timelock, this exact interval value being an implementation and > node policy setting. As a minimal value, the specification recommends 34 > blocks of interval. If the timelock expiration I of the inbound HTLC is > equal to 100 from chain tip, the timelock expiration O of the outbound HTLC > must be equal to 66 blocks from chain tip, giving a reasonable buffer of > reaction to the lightning forwarding node. > > In the lack of cooperative off-chain settlement of the HTLC on the > outgoing link negotiated with the counterparty (either > `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the > lightning node should broadcast its commitment transaction. Once the > commitment is confirmed (if anchor and the 1 CSV encumbrance is present), > the
