Re: [Lightning-dev] [RFC] Lightning gossip alternative
Olaoluwa Osuntokun writes: > Hi Rusty, > >> Timestamps are replaced with block heights. > > This is a conceptually small change, but would actually make things like > rate limiting updates for implementations easier and more uniform. A simple > rule would be only allowing an update per block, which cuts down a lot on > potential chatter traffic, but maybe it's _too_ restrictive? Based on my > network observations, these days some many power user nodes more > aggressively > update their fee schedules in response to liquidity imbalances or as an > attempt to incentive usage of some channels vs others. As an aside, Alex Myers is looking at using minisketch for gossip reconciliation, and there's enough spam in the network that it probably requires some consensus on filtering. When he's got some better analysis I expect a post here... >> 1. Nodes send out weekly node_announcement_v2, proving they own some >> UTXOs. > > If a node doesn't send out this announcement, then will others start to > ignore their "channels"? Yes, this would be the anchor (reverse of now, ignore node_announcement unless you've seen a channel_announcement). >> 3. This uses UTXOs for anti-spam, but doesn't tie them to channels >> directly. > > As I hinted a bit in prior discussion, and also my recent ML [1] post > outlining a possible "do most of the same things" stop gap, this has the > potentially undesirable effect of allowing parties on the network to utilize > _existing_ outputs to advertise false channels and inflate the "total > network capacity" metric. We'd effectively be moving away from "Alice and > Bob have N BTC of bound capacity between them to", "Bob has N BTC he can use > for signing these proofs". Since the aim is to defeat onchain analysis, this seems inevitable in almost any non-zero-knowledge scheme? > Also while we're at it, why not add a merkle proof here (independent of > which direction we go with) which makes it possible for constrained/mobile > nodes to more easily verify gossip data (could be an optional query flag). Yeah, we should do that anyway (the merkle proof doesn't need to be signed, so can be added by anyone). >> FIXME: what about tapscripts? > > Yeah, one side effect of moving to nodes advertising how much BTC they can > sign w/ vs the _actual_ BTC they have in "channels", is that to extend > validation here, the verifiers would need to fully verify possible script > path spends (assuming a scenario where a NUMs point is used as the internal > key). Yech, you're right :( >> Legacy proofs are two signatures, similar to the existing >> channel_announcement. > > Why not musig2? We'd be able to get away with just a single sig with this > modified `node_announcement_v2` and if we go the `channel_announcement2` > route, we'd be able to compress those 4 sigs into one. Good point: I was thinking this would allow nodes to advertize v2 even if their peer doesn't support it, but it doesn't. >> - If two node_announcement_v2 claim the same UTXO, use the first seen, >> discard any others. > > So then this would mean that nodes that _actually_ have a channel between > them can't _individually_ claim the capacity? Yeah, it's kind of arbitrary who gets the UTXO in that case, but for simplicitly I think we need to enforce uniqueness. They can flip a coin? >> - node_announcement_v2 are discarded after a week (1000 blocks). - Nodes >> do not need to monitor existence of UTXOs after initial check (since they >> will automatically prune them after a week). > > A side effect of this would be _continual_ gossip churn in order to keep > channels alive. Today we do have the 2 week `channel_update` heart beat, but > channel updates are relatively small compared to this `node_announcement_v2` > message. We could increase this to 2000 blocks, but #nodes << #channels, so it's still probably a win? I'd have to run some real numbers though... >> - The total proved utxo value (not counting any utxos which are spent) is >> multiplied by 10 to give the "announcable_channel_capacity" for that node. > > I don't see how this is at all useful in practice. We'd end up with inflated > numbers for the total node capacity, and path finding would be more > difficult as it isn't clear exactly how large an HTLC I can send over the > "channel". Sure there's the existence of max_htlc, but in that case why add > this "leverage" factor in the first place? Because not every node will have enough onchain funds, they'll end up revealing some channels' UTXOs. The larger the leverage, the less revealed... >> 1. type: 273 (`channel_update_v2`) > > This seems to allow the advertisement of channels which aren't actually > anchored in the chain, which I *think* is a cool thing to have? On the other > hand, the graph at the _edge_ level would be far more dynamic than it is > today (Bob can advertise an entirely distinct topology from one day to > another). Would need to think about the implications here for path finding > algorithms and n
Re: [Lightning-dev] [RFC] Lightning gossip alternative
Hi Rusty, > Timestamps are replaced with block heights. This is a conceptually small change, but would actually make things like rate limiting updates for implementations easier and more uniform. A simple rule would be only allowing an update per block, which cuts down a lot on potential chatter traffic, but maybe it's _too_ restrictive? Based on my network observations, these days some many power user nodes more aggressively update their fee schedules in response to liquidity imbalances or as an attempt to incentive usage of some channels vs others. > 1. Nodes send out weekly node_announcement_v2, proving they own some > UTXOs. If a node doesn't send out this announcement, then will others start to ignore their "channels"? > 3. This uses UTXOs for anti-spam, but doesn't tie them to channels > directly. As I hinted a bit in prior discussion, and also my recent ML [1] post outlining a possible "do most of the same things" stop gap, this has the potentially undesirable effect of allowing parties on the network to utilize _existing_ outputs to advertise false channels and inflate the "total network capacity" metric. We'd effectively be moving away from "Alice and Bob have N BTC of bound capacity between them to", "Bob has N BTC he can use for signing these proofs". Also while we're at it, why not add a merkle proof here (independent of which direction we go with) which makes it possible for constrained/mobile nodes to more easily verify gossip data (could be an optional query flag). > FIXME: what about tapscripts? Yeah, one side effect of moving to nodes advertising how much BTC they can sign w/ vs the _actual_ BTC they have in "channels", is that to extend validation here, the verifiers would need to fully verify possible script path spends (assuming a scenario where a NUMs point is used as the internal key). > Legacy proofs are two signatures, similar to the existing > channel_announcement. Why not musig2? We'd be able to get away with just a single sig with this modified `node_announcement_v2` and if we go the `channel_announcement2` route, we'd be able to compress those 4 sigs into one. > - If two node_announcement_v2 claim the same UTXO, use the first seen, > discard any others. So then this would mean that nodes that _actually_ have a channel between them can't _individually_ claim the capacity? > - node_announcement_v2 are discarded after a week (1000 blocks). - Nodes > do not need to monitor existence of UTXOs after initial check (since they > will automatically prune them after a week). A side effect of this would be _continual_ gossip churn in order to keep channels alive. Today we do have the 2 week `channel_update` heart beat, but channel updates are relatively small compared to this `node_announcement_v2` message. > - The total proved utxo value (not counting any utxos which are spent) is > multiplied by 10 to give the "announcable_channel_capacity" for that node. I don't see how this is at all useful in practice. We'd end up with inflated numbers for the total node capacity, and path finding would be more difficult as it isn't clear exactly how large an HTLC I can send over the "channel". Sure there's the existence of max_htlc, but in that case why add this "leverage" factor in the first place? > 1. type: 273 (`channel_update_v2`) This seems to allow the advertisement of channels which aren't actually anchored in the chain, which I *think* is a cool thing to have? On the other hand, the graph at the _edge_ level would be far more dynamic than it is today (Bob can advertise an entirely distinct topology from one day to another). Would need to think about the implications here for path finding algorithms and nodes that want to maintain an update to date view of the network... > - `channel_id_and_claimant` is a 31-bit per-node channel_id which can be > used in onion_messages, and a one bit stolen for the `claim` flag. Where would this `channel_id` be derived from? FWIW, using this value in the onion means we get a form of pubkey based routing [3] depending on how these are derived. > This simplifies gossip, requiring only two messages instead of three, > and reducing the UTXO validation requirements to per-node instead of > per-channel. I'm not sure this would actually _simplify_ gossip in practice, given that we'd be moving to a channel graph that isn't entirely based in the reality of what's routable, and would be far more dynamic than it is today. > We can use a convention that a channel_update_v2 with no `capacity` is a > permanent close. On the neutrino side, we tried to do something where if we see both channels be disabled, then we'd mark the channel as closed. But in practice if you're not syncing _every_ channel update every transmitted, then you'll end up actually missing them. > It also allows "leasing" of UTXOs: you could pay someone to sign their > UTXO for your node_announcement, with some level of trust. I'm not sure this is entirely a *good* thing, as the
Re: [Lightning-dev] [RFC] Lightning gossip alternative
ZmnSCPxj writes: > Good morning rusty, > > If we are going to switch to a new gossip version, should we prepare now for > published channels that are backed by channel factories? This is already true with the new proposal: channels don't have to be "real". It's possible to raise the required ratio later, so less UTXO proof is required (since channels are prioritized by their id, you can choose which ones older nodes will see. Cheers, Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [RFC] Lightning gossip alternative
Good morning rusty, If we are going to switch to a new gossip version, should we prepare now for published channels that are backed by channel factories? Instead of a UTXO serving as a bond to allow advertisement of a *single* channel, allow it to advertise *multiple* channels. This does not require us to flesh out the details of channel factories in the gossip protocol, especially post-Taproot --- we could simply require that a simple BIP-340 signature of the gossip message attesting to multiple channels is enough, and the details of the channel factories can be left to later protocol updates. The reason for this is that for a set of N published nodes, there is an incentive to make as many channels as possible between pairs of nodes. We expect that for N published nodes, all (N * (N - 1)) / 2 possible channels will be created, as that maximizes the expected fee return of the N published nodes. Without the ability to gossip channel factories, channel factories can only be used for unpublished channels. Due to not being available for routing, given an "LSP" and N clients, there is no incentive for the N clients to make direct channels to each other. (In particular, one of the reasons given for unpublished channels is that the clients of an LSP may not have high onlineness, thus an unpublished channel would really only exist between a public LSP and a non-published client of that LSP.) This means that for N clients we expect only N channels backed by the channel factory (and thus by the UTXO). It seems to me to be a good idea to have as much of the public network backed by fewer UTXOs, as the published network has far more channels for every N participants. (as well, supporting channel factories for the public graph does not preclude the unpublished graph from using channel factories, so even if the unpublished graph turns out to be much larger than the published graph, reducing the UTXO set of the published graph does not prevent reducing the UTXO set of the unpublished graph anyway.) Against this, we should note that it makes stuffing the public graph cheaper (a single UTXO can now justify the addition of multiple edges on the public routing graph), which translates to making it easier to increase the complexity of the public graph and thus increase the cost of pathfinding. Thoughts? Regards, ZmnSCPxj ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [RFC] Lightning gossip alternative
Joost Jager writes:
> Hi Rusty,
>
> Nice to see the proposal in more concrete terms. Few questions:
>
> - The total proved utxo value (not counting any utxos which are spent)
>> is multiplied by 10 to give the "announcable_channel_capacity" for that
>> node.
>>
>
> Could this work as a dynamic value too, similar to the minimum relay fee on
> L1?
I made the number up, so I'm not very attached to it. How would we
choose to scale it though? If nodes don't use the same value it makes
for wasted gossip (and minisketch-based gossip much harder).
>> 1. `tlv_stream`: `channel_update_v2_tlvs`
>>
> 2. types:
>> 1. type: 4 (`capacity`)
>> 2. data:
>> * [`tu64`:`satoshis`]
>>
>
> What does capacity mean exactly outside the context of a real channel? Will
> this be reduced to that maximum htlc amount that the nodes want to route,
> to save as much of the announceable budget as possible?
Yes, it's the old htlc_maximum_msat, but expressed in satoshis because
msat seems to finegrained?
> It is also the question of whether 10 x 10k channels should weigh as much
> on the budget as a 1 x 100k channel. A spammer may be able to do more harm
> with multiple smaller channels because there is more for the sender's
> pathfinding algorithms to explore. Maybe it doesn't matter as long as there
> is some mechanism to discourage spam.
Yes, I suggested a minimum cost to avoid 100k 1sat channels, I don't
know if we should be more sophisticated.
>> 1. type: 5 (`cost`)
>> 2. data:
>>* [`u16`:`cltv_expiry_delta`]
>>* [`u32`:`fee_proportional_millionths`]
>>* [`tu32`:`fee_base_msat`]
>> 1. type: 6 (`min_msat`)
>> 2. data:
>> * [`tu64`:`min_htlc_sats`]
>>
>> - `channel_id_and_claimant` is a 31-bit per-node channel_id which can be
>> used in onion_messages, and a one bit stolen for the `claim` flag.
>
> If you'd increase the budget multiplier from 10 to 20, couldn't this be
> simplified to always applying the cost to both nodes?
Yes! I forgot that capacity doesn't have to be symmetrical; if I open a
giant channel with you, and you don't want to expose more UTXOs, you
just set your capacity to some lower value.
>> - A channel is not considered to exist until both peers have sent a
>> channel_update_v2, at least one of which must set the `claim` flag.
>> - If a node sets `claim`, the capacity of the channel is subtracted from
>> the remaining announcable_channel_capacity for that node (minimum
>> 10,000 sats).
>
> Same question about magic value and whether it can be dynamic.
Yes, 10ksat might be giant one day? We can change it naively with a
feature bit ("I use v2 capacity calcs"), or in some more finegrained
way, but what do we base it on?
Cheers!
Rusty.
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [RFC] Lightning gossip alternative
Hi Rusty, Nice to see the proposal in more concrete terms. Few questions: - The total proved utxo value (not counting any utxos which are spent) > is multiplied by 10 to give the "announcable_channel_capacity" for that > node. > Could this work as a dynamic value too, similar to the minimum relay fee on L1? > 1. `tlv_stream`: `channel_update_v2_tlvs` > 2. types: > 1. type: 4 (`capacity`) > 2. data: > * [`tu64`:`satoshis`] > What does capacity mean exactly outside the context of a real channel? Will this be reduced to that maximum htlc amount that the nodes want to route, to save as much of the announceable budget as possible? It is also the question of whether 10 x 10k channels should weigh as much on the budget as a 1 x 100k channel. A spammer may be able to do more harm with multiple smaller channels because there is more for the sender's pathfinding algorithms to explore. Maybe it doesn't matter as long as there is some mechanism to discourage spam. > 1. type: 5 (`cost`) > 2. data: >* [`u16`:`cltv_expiry_delta`] >* [`u32`:`fee_proportional_millionths`] >* [`tu32`:`fee_base_msat`] > 1. type: 6 (`min_msat`) > 2. data: > * [`tu64`:`min_htlc_sats`] > > - `channel_id_and_claimant` is a 31-bit per-node channel_id which can be > used in onion_messages, and a one bit stolen for the `claim` flag. > If you'd increase the budget multiplier from 10 to 20, couldn't this be simplified to always applying the cost to both nodes? > - A channel is not considered to exist until both peers have sent a > channel_update_v2, at least one of which must set the `claim` flag. > - If a node sets `claim`, the capacity of the channel is subtracted from > the remaining announcable_channel_capacity for that node (minimum > 10,000 sats). > Same question about magic value and whether it can be dynamic. Joost ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
[Lightning-dev] [RFC] Lightning gossip alternative
Hi all, I've floated this idea before, but this is a more concrete proposal for a "v2" gossip protocol. It assumes x-only pubkeys (aka point32) and BIP-340 signatures, and uses modern TLVs for all optional or extensible fields. Timestamps are replaced with block heights. 1. Nodes send out weekly node_announcement_v2, proving they own some UTXOs. 2. This entitles them to broadcast channels, using channel_update_v2; a channel_update_v2 from both peers means the channel exists. 3. This uses UTXOs for anti-spam, but doesn't tie them to channels directly. 4. Future ZKP proofs are could be added. 1. type: 271 (`node_announcement_v2`) 2. data: * [`bip340sig`:`signature`] * [`point32`:`node_id`] * [`u32`:`blockheight`] * [`node_announcement_v2_tlvs`:`tlvs`] 1. `tlv_stream`: `node_announcement_v2_tlvs` 2. types: 1. type: 2 (`features`) 2. data: * [`...*byte`:`featurebits`] 1. type: 3 (`chain_hash`) 2. data: * [`chain_hash`:`chain`] 1. type: 4 (`taproot_utxo_proofs`) 2. data: * [`...*taproot_utxo_proof`:`proofs`] 1. type: 6 (`legacy_utxo_proofs`) 2. data: * [`...*legacy_utxo_proof`:`proofs`] 1. type: 127 (`ipv4_addresses`) 2. data: * [`...*ipv4`:`addresses`] 1. type: 129 (`ipv6_addresses`) 2. data: * [`...*ipv6`:`addresses`] 1. type: 131 (`torv3_addresses`) 2. data: * [`...*torv3`:`addr`] # Maybe alias, color, etc? Taproot proofs are a signature of the v1 output over the `node_id`, `utxo` and `blockheight` with prefix "lightingtaproot_utxo_proofsig" (using the tagged signatures as per BOLT12). (FIXME: what about tapscripts?). Legacy proofs are two signatures, similar to the existing channel_announcement. 1. subtype: `taproot_utxo_proof` 2. data: * [`short_channel_id`:`utxo`] * [`signature`:`sig`] 1. subtype: `legacy_utxo_proof` 2. data: * [`short_channel_id`:`utxo`] * [`point`:`bitcoin_key_1`] * [`point`:`bitcoin_key_2`] * [`signature`:`sig_1`] * [`signature`:`sig_2`] - node_announcement_v2 are discarded after a week (1000 blocks). - If two node_announcement_v2 claim the same UTXO, use the first seen, discard any others. - Nodes do not need to monitor existence of UTXOs after initial check (since they will automatically prune them after a week). - The total proved utxo value (not counting any utxos which are spent) is multiplied by 10 to give the "announcable_channel_capacity" for that node. 1. type: 273 (`channel_update_v2`) 2. data: * [`bip340sig`:`signature`] * [`point32`:`local_node_id`] * [`point32`:`remote_node_id`] * [`u32`:`blockheight`] * [`u32`:`channel_id_and_claimant`] * [`channel_update_v2_tlvs`:`tlvs`] 1. `tlv_stream`: `channel_update_v2_tlvs` 2. types: 1. type: 2 (`features`) 2. data: * [`...*byte`:`featurebits`] 1. type: 3 (`chain_hash`) 2. data: * [`chain_hash`:`chain`] 1. type: 4 (`capacity`) 2. data: * [`tu64`:`satoshis`] 1. type: 5 (`cost`) 2. data: * [`u16`:`cltv_expiry_delta`] * [`u32`:`fee_proportional_millionths`] * [`tu32`:`fee_base_msat`] 1. type: 6 (`min_msat`) 2. data: * [`tu64`:`min_htlc_sats`] - `channel_id_and_claimant` is a 31-bit per-node channel_id which can be used in onion_messages, and a one bit stolen for the `claim` flag. - A channel is not considered to exist until both peers have sent a channel_update_v2, at least one of which must set the `claim` flag. - If a node sets `claim`, the capacity of the channel is subtracted from the remaining announcable_channel_capacity for that node (minimum 10,000 sats). - If there is insufficient total `announcable_channel_capacity` for a node, it is used by the lower `channel_id`s first. Implications This simplifies gossip, requiring only two messages instead of three, and reducing the UTXO validation requirements to per-node instead of per-channel. We can use a convention that a channel_update_v2 with no `capacity` is a permanent close. We might want to add a taproot_utxo_delegated_proof where UTXOs can sign over to another key, so cold storage only needs to sign once, and the other key can sign weekly. It also allows "leasing" of UTXOs: you could pay someone to sign their UTXO for your node_announcement, with some level of trust. They could spend the UTXO, which gives you a slow degredation as new nodes don't accept your channels but existing nodes don't check until it's due for a refresh). Or they could sign one UTXO for multiple node_announcements, which is why preference is given to the first-seen. But it's a weekly business so there's incentive for them not to. Between nodes there's the question of "who claims this new channel?", which I didn't address here. Opener claims is logical, but leaks information (though you could definitely pay for the peer to claim it). With dual-funding, it's more comple
