Re: [Lightning-dev] Rebalancing argument
Good morning, I believe care must be taken for automatic rebalancing. Suppose we start in this state: A 1.0|\1.0 | \ 1.0| \1.0 B---C 1.0 1.0 Then A pays to B 1.0 BTC: A 0.0|\1.0 | \ 2.0| \1.0 B---C 1.0 1.0 In an effort to balance, B moves 0.5 from A-B to B-C: A 0.5|\0.5 | \ 1.5| \1.5 B---C 1.5 0.5 In an effort to balance, C moves 0.5 from A-C to C-B: A 0.0|\1.0 | \ 2.0| \1.0 B---C 1.0 1.0 Which again triggers B to balance, setting both of them up to fight each other while A just charges them fees on the payments both ways. Eventually it settles with A recouping the entire 1.0 it paid to B from the fees of both B and C fighting things out between them about keeping their channels balanced. Regards, ZmnSCPxj Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐ Original Message ‐‐‐ On July 10, 2018 6:23 PM, Johan Torås Halseth wrote: > A simple way to see how rebalancing could be beneficial, is to observe that > you only know the channel capacity (not the balance!) of the channels you > don’t belong to. > > If everybody is being good stewards and are rebalancing their channels, then > picking a route to send a payment over is more likely to succeed than if > everybody only has channels depleted in one direction (extreme case). > > On Thu, Jul 5, 2018 at 12:06, Dmytro Piatkivskyi > wrote: > >> Hi Olaoluwa, >> >> I¹m glad we¹re thinking the same direction. >> >> Generally, I think we (the community) worry too much about equilibrium. >> There is no really proof that it improves the network. On the other hand, >> money being locked in channel is of major issue. Some nodes may be used >> mostly for sending payments, whereas others mostly receiving. Therefore, >> the distribution of funds in channels should be dictated not by >> equilibrium but by nodes' plans to send and receive. >> >>> In this case, equilibrium means being able to recv as much as you can >>>send on all your channels. >> >> Now it seems there are two ways to define equilibrium. You have described >> one where each node is trying to keep the ability to send and receive at >> the same level. I¹ll repeat the above argument, some nodes may be used >> mostly for sending payments, whereas others mostly receiving. Therefore, >> such definition is unjustified from the individual viewpoint. Another >> definition of equilibrium is when a node distributes equally the available >> balance amongst the channels it has. Your argument still stands here as >> such equilibrium minimises the number of depleted channels. >> >>> The argument here is that by maintaining this balance, one is likely to >>>reduce the number of routing failures from failed HTLC forwarding, as the >>>channel is equally likely to be able to carry an HTLC in either direction. >> >> If a node has no balance, its channels are depleted. There is nothing we >> can do with this. Such nodes are bad for topology and should be >> discouraged. Possibly by the autopilot. >> >>> However if a few sources/sinks dominate, then channels may regularly >>>become biased requiring more maintenance. >> >> Now you¹re bringing up even more important question. If we had balanced >> payments, LN could function without touching blockchain ever again >> indefinitely. Skewed traffic is a big problem. Re-balancing won¹t be of >> use here because having a fixed nodes¹ balances, there is only a limited >> max flow that can be pushed in a particular direction. I believe autopilot >> could mitigate the problem. Please, find my argument at the bottom of [1]. >> >> [1] https://github.com/lightningnetwork/lnd/issues/677 >> >> Best, >> Dmytro >> >> From: Olaoluwa Osuntokun >> Date: Tuesday, 3 July 2018 at 22:13 >> To: Dmytro Piatkivskyi >> Cc: "lightning-dev@lists.linuxfoundation.org" >> >> Subject: Re: [Lightning-dev] Rebalancing argument >> >> Hi Dmytro, >> >> Thanks for bringing this up! Sometime last year when I was at a developer >> meetup that cdecker also attended, we briefly discussed a similar >> question. We >> we're discussing if "active rebalancing" was actually ever really >> necessary. >> From my PoV, active rebalancing is rebalancing done for the purpose of >> being >> able to send/recv a particular payment. On white board, cdecer sketched >> out a >> similar argument as to Lemma 2 in that paper you linked: namely that there >> will >> exist an alternative path, therefore act
Re: [Lightning-dev] Rebalancing argument
A simple way to see how rebalancing could be beneficial, is to observe that you only know the channel capacity (not the balance!) of the channels you don’t belong to. If everybody is being good stewards and are rebalancing their channels, then picking a route to send a payment over is more likely to succeed than if everybody only has channels depleted in one direction (extreme case). On Thu, Jul 5, 2018 at 12:06, Dmytro Piatkivskyi wrote: Hi Olaoluwa, I¹m glad we¹re thinking the same direction. Generally, I think we (the community) worry too much about equilibrium. There is no really proof that it improves the network. On the other hand, money being locked in channel is of major issue. Some nodes may be used mostly for sending payments, whereas others mostly receiving. Therefore, the distribution of funds in channels should be dictated not by equilibrium but by nodes' plans to send and receive. > In this case, equilibrium means being able to recv as much as you can >send on all your channels. Now it seems there are two ways to define equilibrium. You have described one where each node is trying to keep the ability to send and receive at the same level. I¹ll repeat the above argument, some nodes may be used mostly for sending payments, whereas others mostly receiving. Therefore, such definition is unjustified from the individual viewpoint. Another definition of equilibrium is when a node distributes equally the available balance amongst the channels it has. Your argument still stands here as such equilibrium minimises the number of depleted channels. > The argument here is that by maintaining this balance, one is likely to >reduce the number of routing failures from failed HTLC forwarding, as the >channel is equally likely to be able to carry an HTLC in either direction. If a node has no balance, its channels are depleted. There is nothing we can do with this. Such nodes are bad for topology and should be discouraged. Possibly by the autopilot. > However if a few sources/sinks dominate, then channels may regularly >become biased requiring more maintenance. Now you¹re bringing up even more important question. If we had balanced payments, LN could function without touching blockchain ever again indefinitely. Skewed traffic is a big problem. Re-balancing won¹t be of use here because having a fixed nodes¹ balances, there is only a limited max flow that can be pushed in a particular direction. I believe autopilot could mitigate the problem. Please, find my argument at the bottom of [1]. [1] https://github.com/lightningnetwork/lnd/issues/677 Best, Dmytro From: Olaoluwa Osuntokun Date: Tuesday, 3 July 2018 at 22:13 To: Dmytro Piatkivskyi Cc: "lightning-dev@lists.linuxfoundation.org" Subject: Re: [Lightning-dev] Rebalancing argument Hi Dmytro, Thanks for bringing this up! Sometime last year when I was at a developer meetup that cdecker also attended, we briefly discussed a similar question. We we're discussing if "active rebalancing" was actually ever really necessary. >From my PoV, active rebalancing is rebalancing done for the purpose of being able to send/recv a particular payment. On white board, cdecer sketched out a similar argument as to Lemma 2 in that paper you linked: namely that there will exist an alternative path, therefore active rebalancing isn't necessary. IMO, in a world pre-AMP, it is indeed necessary. Consider a node Bob that wishes to receive a payment of 0.5 BTC. Bob has two channels, one with 2 BTC max capacity, and the other with 1 BTC max capacity. If channel 1 only has 0.2 available for him to receive, and channel 2 only has 0.3 available for him to receive, then without active rebalancing, he's unable to receive the payment. However, if he completes a circular payment from channel 1 to channel 2 (or the other way around), then he's able to receive the payment (under ideal conditions). In a world post-AMP, then this would seem to no longer apply. Alice the sender may be able to utilize the aggregate bandwidth of the network to send minimally two payment flows (one 0.2 and one 0.3) to satisfy the payment. As a result, active rebalancing isn't needed as payments can be split up to fully utilize the payment bandwidth at both the sender and the receiver. > total balances of nodes in the network define the feasibility of a >particular > transaction, not the distribution of balances. With multi-path payments this is precisely the case! However, there might be an argument in favor of "passive rebalancing". I define passive rebalancing as rebalancing a node carries out independent of needing to send/receive payments themselves, in order to ensure an equilibrium state amongst the available balances of their channels. In this case, equilibrium means being able to recv as much as you can send on all your channels. The argument here is that by maintaining this balance, one is likely to reduce the number
Re: [Lightning-dev] Rebalancing argument
Hi Olaoluwa, I¹m glad we¹re thinking the same direction. Generally, I think we (the community) worry too much about equilibrium. There is no really proof that it improves the network. On the other hand, money being locked in channel is of major issue. Some nodes may be used mostly for sending payments, whereas others mostly receiving. Therefore, the distribution of funds in channels should be dictated not by equilibrium but by nodes' plans to send and receive. > In this case, equilibrium means being able to recv as much as you can >send on all your channels. Now it seems there are two ways to define equilibrium. You have described one where each node is trying to keep the ability to send and receive at the same level. I¹ll repeat the above argument, some nodes may be used mostly for sending payments, whereas others mostly receiving. Therefore, such definition is unjustified from the individual viewpoint. Another definition of equilibrium is when a node distributes equally the available balance amongst the channels it has. Your argument still stands here as such equilibrium minimises the number of depleted channels. > The argument here is that by maintaining this balance, one is likely to >reduce the number of routing failures from failed HTLC forwarding, as the >channel is equally likely to be able to carry an HTLC in either direction. If a node has no balance, its channels are depleted. There is nothing we can do with this. Such nodes are bad for topology and should be discouraged. Possibly by the autopilot. > However if a few sources/sinks dominate, then channels may regularly >become biased requiring more maintenance. Now you¹re bringing up even more important question. If we had balanced payments, LN could function without touching blockchain ever again indefinitely. Skewed traffic is a big problem. Re-balancing won¹t be of use here because having a fixed nodes¹ balances, there is only a limited max flow that can be pushed in a particular direction. I believe autopilot could mitigate the problem. Please, find my argument at the bottom of [1]. [1] https://github.com/lightningnetwork/lnd/issues/677 Best, Dmytro From: Olaoluwa Osuntokun Date: Tuesday, 3 July 2018 at 22:13 To: Dmytro Piatkivskyi Cc: "lightning-dev@lists.linuxfoundation.org" Subject: Re: [Lightning-dev] Rebalancing argument Hi Dmytro, Thanks for bringing this up! Sometime last year when I was at a developer meetup that cdecker also attended, we briefly discussed a similar question. We we're discussing if "active rebalancing" was actually ever really necessary. >From my PoV, active rebalancing is rebalancing done for the purpose of being able to send/recv a particular payment. On white board, cdecer sketched out a similar argument as to Lemma 2 in that paper you linked: namely that there will exist an alternative path, therefore active rebalancing isn't necessary. IMO, in a world pre-AMP, it is indeed necessary. Consider a node Bob that wishes to receive a payment of 0.5 BTC. Bob has two channels, one with 2 BTC max capacity, and the other with 1 BTC max capacity. If channel 1 only has 0.2 available for him to receive, and channel 2 only has 0.3 available for him to receive, then without active rebalancing, he's unable to receive the payment. However, if he completes a circular payment from channel 1 to channel 2 (or the other way around), then he's able to receive the payment (under ideal conditions). In a world post-AMP, then this would seem to no longer apply. Alice the sender may be able to utilize the aggregate bandwidth of the network to send minimally two payment flows (one 0.2 and one 0.3) to satisfy the payment. As a result, active rebalancing isn't needed as payments can be split up to fully utilize the payment bandwidth at both the sender and the receiver. > total balances of nodes in the network define the feasibility of a >particular > transaction, not the distribution of balances. With multi-path payments this is precisely the case! However, there might be an argument in favor of "passive rebalancing". I define passive rebalancing as rebalancing a node carries out independent of needing to send/receive payments themselves, in order to ensure an equilibrium state amongst the available balances of their channels. In this case, equilibrium means being able to recv as much as you can send on all your channels. The argument here is that by maintaining this balance, one is likely to reduce the number of routing failures from failed HTLC forwarding, as the channel is equally likely to be able to carry an HTLC in either direction. One relevant detail w.r.t to the necessity of active rebalancing is the directionality of payment flows in the network. If payment flows are more or less balanced (kinda like the ideal world Byran Vu describes in the post [1]), meaning people are sending out as much as they receive (so they get their paycheck st
Re: [Lightning-dev] Rebalancing argument
Dmytro wrote: > Yet the question remains — are there obvious advantages of cycle > transactions over a smart fee/routing system? That's a good question, it may be the case that by modifying the fee structure to punish flows that unbalance channels further, then this can simplify the problem as the heuristics can target solely the fee rate. The earliest suggestion of this I can recall was from Tadge way back in like 2015 or so. The goal here is for a node to ideally maintain relatively balanced channels, but then charge a payment an amount that scales super linearly when flows consume most of their available balance. The current fee schedule is essentially: base_fee + amt*rate clighting and lnd (borrowed from c-lightning) currently use a "risk factor" to factor in the impact of the time lock on the "weight" of an edge when path finding. With this, the fee schedule looks like: (base_fee + amt*rate) + (cltv_delta * risk_factor / 1,000,) In the future, we may want to allow nodes to also signal how they wish the fee to scale with the absolute CLTV of the HTLC extend to this. This would allow them to more naturally factor in their conception of the time value of their BTC. Finally, if we factor in an "balance disruption" factor, the fee schedule may look something like this: (base_fee + amt*rate) + (cltv_delta * risk_factor / 1,000,) + gamma*f(capacity, amt) Here f is a function whose output is proportional to the distance the payment flow (assuming full capacity at that instance) puts the channel away from being fully balanced, and gamma is a coefficient that allows nodes to express the degree of penalty for unbalancing a channel. f itself is either agreed upon by the network completely, or resembles a certain polynomial, allowing nodes to select coefficients as they wish. We may want to consider moving to something like this for BOLT 1.x+ as it allows nodes to quantify their apprehension to time locks and also channel balance equilibrium affinity. Alternatively, if we move to something like HORNET, then during the set up phase, nodes can ask for an initial "quote" for a set of payment ranges, then just use that quote for all payments sent. This allows nodes to keep their fee schedules private (for w/e reason) and also change them at a whim if they wish. -- Laolu On Sun, Jul 1, 2018 at 8:39 AM Dmytro Piatkivskyi < dmytro.piatkivs...@ntnu.no> wrote: > Hi Rene, > > Thanks for your answer! > > 1. The Lightning network operates under different assumptions, that is > true. However, the difference is minor in terms of the issue posed. The > premise for the quoted statement is that taking fees changes the nodes’ > balances, therefore selected paths affect the liquidity. In the Lightning > network fees are very small, so the change in liquidity may be negligible. > Moreover, intermediate nodes gain in fees, which only increases the > liquidity. > > 2.A. It is too early to speculate where the privacy requirements will > settle down. Flare suggests a mechanism of sharing the infrastructure view > between nodes, possibly sharing weights. As the network grows routing will > become more difficult, however we don’t know yet to which extent. It may > organise itself in ‘domains’, so when we send a payment we know to which > domain we are sending to, knowing the path to it beforehand. The point is > we don’t know yet, so we can’t speculate. > > 2.B. That is surely an interesting aspect. HTLC locks > temporarily downgrade the network liquidity. Now the question is how it > changes the order of transactions and how that order change affects the > transaction feasibility. Does it render some transactions infeasible or > just defers them? It definitely needs a closer look. > > Yet the question remains — are there obvious advantages of cycle > transactions over a smart fee/routing system? In any sense. Path lengths, > for example. To answer that I am going to run a simulation, but also would > appreciate your opinions. > > Best, > Dmytro > > From: René Pickhardt > Date: Sunday, 1 July 2018 at 13:59 > To: Dmytro Piatkivskyi > Cc: lightning-dev > Subject: Re: [Lightning-dev] Rebalancing argument > > Hey Dmytro, > > thank your for your solid input to the discussion. I think we need to > consider that the setting in the lightning network is not exactly > comparable to the one described in the 2010 paper. > > 1st: the paper states in section 5.2: "It appears that a mathematical > analysis of a transaction routing model where intermediate nodes charged a > routing fee would require an entirely new approach since it would > invalidate the cycle-reachability relation that forms the basis of our > results." > Since we have routing fees in the lightning network to my understandi
