Re: [Lightning-dev] Highly Available Lightning Channels

2023-02-15 Thread Matt Corallo



On 2/14/23 11:36 PM, Joost Jager wrote:

But how do you decide to set it without a credit relationship? Do I measure 
my channel and set the

bit because the channel is "usually" (at what threshold?) saturating in the 
inbound direction? What
happens if this changes for an hour and I get unlucky? Did I just screw 
myself?


As a node setting the flag, you'll have to make sure you open new channels, rebalance or swap-in in 
time to maintain outbound liquidity. That's part of the game of running an HA channel.


Define "in time" in a way that results in senders not punishing you for not meeting your "HA 
guarantees" due to a large flow. I don't buy that this results in anything other than pressure to 
add credit.



 > How can you be sure about this? This isn't publicly visible data.

Sure it is! https://river.com/learn/files/river-lightning-report.pdf



Some operators publish data, but are the experiences of one of the most well connected (custodial) 
nodes representative for the network as a whole when evaluating payment success rates? In the end 
you can't know what's happening on the lightning network.


Right, that was my above point about fetching scoring data - there's three relevant "buckets" of 
nodes, I think - (a) large nodes sending lots of payments, like the above, (b) "client nodes" that 
just connect to an LSP or two, (c) nodes that route some but don't send a lot of payments (but do 
send *some* payments), and may have lots or not very many channels.


(a) I think we're getting there, and we don't need to add anything extra for this use-case beyond 
the network maturing and improving our scoring algorithms.
(b) I think is trivially solved by downloading the data from a node in category (a), presumably the 
LSP(s) in question (see other branch of this thread)
(c) is trickier, but I think the same solution of just fetching semi-trusted data here more than 
sufficies. For most routing nodes that don't send a lot of payments we're talking about a very small 
amount of payments, so trusting a third-party for scoring data seems reasonable.


Once we do that, everyone gets a similar experience as the River report :).

Matt
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] Highly Available Lightning Channels

2023-02-15 Thread Joost Jager
>
> I think the performance question depends on the type of payment flows
> considered. If you're an
> end-user sending a payment to your local Starbucks for coffee, here fast
> payment sounds the end-goal.
> If you're doing remittance payment, cheap fees might be favored, and in
> function of those flows you're
> probably not going to select the same "performant" routing nodes. I think
> adding latency as a criteria for
> pathfinding construction has already been mentioned in the past for LDK
> [0].
>

My hopes are that eventually lightning nodes can run so efficient that in
practice there is no real trade-off anymore between cost and speed. But of
course hard to say how that's going to play out. I am all for adding
latency as an input to pathfinding. Attributable errors should help with
that too.


> Or there is the direction to build forward-error-correction code on top of
> MPP, like in traditional
> networking [1]. The rough idea, you send more payment shards than the
> requested sum, and then
> you reveal the payment secrets to the receiver after an onion
> interactivity round to finalize payment.
>

This is not very different from payment pre-probing is it? So try a larger
set of possible routes simultaneously and when one proves to be open, send
the real payment across that route. Of course a balance may have shifted in
the mean time, but seems unlikely enough to prevent the approach from being
usable. The obvious downside is that the user needs more total liquidity to
have multiple htlcs outstanding at the same time. Nevertheless an
interesting way to reduce payment latency.


> At the end of the day, we add more signal channels between HTLC senders
> and the routing
> nodes offering capital liquidity, if the signal mechanisms are efficient,
> I think they should lead
> to better allocation of the capital. So yes, I think more liquidity might
> be used by routing nodes
> to serve finely tailored HTLC requests by senders, however this liquidity
> should be rewarded
> by higher routing fees.
>

This is indeed part of the idea. By signalling HA, you may not only attract
more traffic, but also be able to command a higher fee.


> I think if we have lessons to learn on policy rules design and deployment
> on the base-layer
> (the full-rbf saga), it's to be careful in the initial set of rules, and
> how we ensure smooth
> upgradeability, from one version to another. Otherwise the re-deployment
> cost towards
> the new version might incentive the old routing node to stay on the
> non-optimal versions,
> and as we have historical buckets in routing algorithms, or preference for
> older channels,
> this might lead the end-user to pay higher fees, than they could access to.
>

I see the parallel, but also it seems that we have this situation already
today on lightning. Senders apply penalties and routing nodes need to make
assumptions about how they are penalised. Perhaps more explicit signalling
can actually help to reduce the degree of uncertainty as to how a routing
nodes is supposed to perform to keep senders happy?


> This is where the open question lies to me - "highly available" can be
> defined with multiple
> senses, like fault-tolerance, latency processing, equilibrated liquidity.
> And a routing node might
> not be able to optimize its architecture for the same end-goal (e.g more
> watchtower on remote
> host probably increases the latency processing).
>

Yes, good point. So maybe a few more bits to signal what a sender may
expect from a channel exactly?


> > Without shadow channels, it is impossible to guarantee liquidity up to
> the channel capacity. It might make sense for senders to only assume high
> > availability for amounts up to `htlc_maximum_msat`.
>
> As a note, I think "senders assumption" should be well-documented,
> otherwise there will be
> performance discrepancies between node implementations or even versions.
> E.g, an upgraded
> sender penalizing a node for the lack of shadow/parallel channels
> fulfilling HTLC amounts up to
> `htlc_maximum_msat`.
>

Well documented, or maybe even explicit in the name of the feature bit. For
example `htlc_max_guaranteed`.


> I think signal availability should be explicit rather than implicit. Even
> if it's coming with more
> gossip bandwidth data consumed. I would say for bandwidth performance
> management, relying
> on new gossip messages, where they can be filtered in function of the
> level of services required
> is interesting.
>

In terms of implementation, I think this kind of signalling is easier as an
extension of `channel_update`, but it can probably work as a separate
message too.

Joost
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev