Re: [Lightning-dev] negative fees for HTLC relay

[Benjamin Mord]

Although lightning and cryptocurrency is new, the idea of a distribution
network created from links with negotiated fees and with limited
unidirectional capacity that can be corrected via rebalancing, is not new.
In fact there are several very large and mature markets around the world
that we can study which illustrate what people have found to work well in
such contexts, and one class of such markets is wholesale electricity
distribution, such as PJM:
http://www.pjm.com/about-pjm/who-we-are.aspx

When Will Yager mentioned FTRs, he was referring to a specific financial
instrument traded on the PJM, for the purpose of managing load on
transmission lines. Much as with lightning payment channels, power flows in
one direction on a line can be canceled with power flows in the opposite
direction, and so FTR prices are therefore allowed to go negative on the
PJM exchanges for the same reasons as discussed in this thread. If this has
worked for them for some decades, I'm not sure why it wouldn't work for us
also.

In full disclosure, I should point out that this is not the same thing as
electricity transmission. Although they do have counterflow cancellation in
common, payment channel balance is still more analogous to energy than
power, whereas transmission capacity limits are in power and not energy.
Time scales are long enough for manual intervention in PJM negotiations,
whereas timescales necessitate a high degree of automation in lightning
negotiation, as another difference. And there are other differences, and
some might matter.

At at a technical level, I see no complexity at all in allowing fees to go
negative. What is hard about allowing signed versus unsigned integers in
the protocol, and even passing them along? Perhaps there is complexity in a
routing algorithm which attempts to take full advantage of negative fees,
and so perhaps implementations will prefer in the near term to pretend
negative fees are actually zero - but such implementations can easily pass
along info about negative fees even when they themselves choose not to
fully avail themselves of the resulting opportunities. (At worst, some
recipients might receive an unexpected tip! There are worse fates in life.)

As relates to lightning's design goals, I applaud simplifying assumptions
in the early implementations, as a matter of triage - but I would still
discourage premature neutering of underlying protocol expressiveness, since
the resulting limitations can then linger or even come to motivate harmful
or risky forks down the road. Passingly along signed instead of unsigned
ints is easy enough, and we can just cast to unsigned internally, wherever
that may be our local preference.

Incidentally, if anyone is interested in exploring specific application of
lightning to the electricity markets, please contact me offline at
b...@mord.io, as it happens I am attempting to organize such an effort.
While I see opportunity here for collaborative and complimentary work that
could in turn boost lightning adoption, I want to be careful also not to
distract the lightning project from its already ambitious and laudable
mission. Lightning must beware scope creep of a harmful sort, and I'll try
to be disciplined in not encouraging it. We should still walk before we run.


On Thu, Jan 18, 2018 at 12:17 PM, Christian Decker <
decker.christ...@gmail.com> wrote:

> Mark Friedenbach  writes:
>
> > It is not the case that all instances where you might have negative
> > fees would have loops.
>
> If we don't have a cycle we can hardly talk about rebalancing
> channels. At that point you're paying for someone else's payment to go
> through your channel, and I'm unclear what the motivation for that might
> be. Anyway, this is still possible by communicating this out of band
> with the payment creator, and should not be baked into the gossip
> protocol itself, in my opinion. It's obscure enough to not be worth the
> extra effort.
>
> > One instance where you want this feature is when the network becomes
> > too weighted in one side of the graph.
>
> There is little you can do to prevent this: if we have a network with a
> small cut, with a source and sink on opposite sides of that cut, no
> amount of voluntary sacrifice from the nodes along the cut will have a
> lasting effect. The better solution would be to change the topology of
> the network to remove the cut, or balance traffic over it, e.g., moving
> a sink to the other side of the cut.
>
> > Another is when the other side is a non-routable endpoint. In both
> > cases would be useful to signal to others that you were willing to pay
> > to rebalance, and this hand wavy argument about loops doesn’t seem to
> > apply.
>
> I'm not sure I understand what you mean with non-routable endpoint, so
> correct me if I'm wrong. I'm assuming that non-routable endpoint is a
> non-publicly announced node in the network? In that case no fee tricks
> will ever get people to route over it, since they 

Re: [Lightning-dev] negative fees for HTLC relay

[Christian Decker]

Mark Friedenbach  writes:

> It is not the case that all instances where you might have negative
> fees would have loops.

If we don't have a cycle we can hardly talk about rebalancing
channels. At that point you're paying for someone else's payment to go
through your channel, and I'm unclear what the motivation for that might
be. Anyway, this is still possible by communicating this out of band
with the payment creator, and should not be baked into the gossip
protocol itself, in my opinion. It's obscure enough to not be worth the
extra effort.

> One instance where you want this feature is when the network becomes
> too weighted in one side of the graph.

There is little you can do to prevent this: if we have a network with a
small cut, with a source and sink on opposite sides of that cut, no
amount of voluntary sacrifice from the nodes along the cut will have a
lasting effect. The better solution would be to change the topology of
the network to remove the cut, or balance traffic over it, e.g., moving
a sink to the other side of the cut.

> Another is when the other side is a non-routable endpoint. In both
> cases would be useful to signal to others that you were willing to pay
> to rebalance, and this hand wavy argument about loops doesn’t seem to
> apply.

I'm not sure I understand what you mean with non-routable endpoint, so
correct me if I'm wrong. I'm assuming that non-routable endpoint is a
non-publicly announced node in the network? In that case no fee tricks
will ever get people to route over it, since they can't even construct
the onion to talk to it. Notice that the payment requests allow for
recipients of payments to get paid by explicitly including the necessary
information to construct the onion to talk to that node.

Not trying to be dismissive here, and I might be getting this wrong, so
let me know if I did and what use-cases you had in mind :-)

Cheers,
Christian
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] negative fees for HTLC relay

[Christian Decker]

Benjamin Mord  writes:
> It isn't obvious to me from the BOLTs if fees can be negative, and I'm
> finding uint in the go source code - which suggests not. In scenarios where
> the funding of a payment channel has been fully committed in one direction,
> why not allow negative fees to incent unwinding, in scenarios where nodes
> consider that cheaper than on-chain rebalancing?

After discussing this for a while we decided not to allow negative fees
in channel announcements (for now), because they actually do not add to
the flexibility and require special handling for route finding.

The main argument for negative fees has always been that they allow a
channel operator to rebalance its channels. However it is neither
required, nor is it really all that helpful. If a node wants to
rebalance he needs to find a cycle, that it can use to rebalance.  The
simplest rebalancing is that the node itself sends a payment along that
cycle back to itself, giving the rebalancing node full control over the
amount to rebalance, timing and costs.

The negative fees were intended to encourage other participants to use
any cycle and rebalance for the node offering the negative fees. However
that results in less control over the rebalancing for the node, e.g.,
how many payments to incentivize, amounts, etc. This is compounded by
the inherent delay of channel updates being disseminated in the
network. So if a rebalancing node gets too many payments that try to
take advantage of the negative fees, what should it do? It'd result in
either losses for the node, or many forward rejections. So why not use
the funds one would have used towards negative fees for the active way
of rebalancing.

It is preferable to have payments be routed around an exhausted channel,
after all if there is a cycle there must be an alternative route, rather
than trying to artificially rebalance.

So overall, allowing only positive fees makes routing simpler, and still
allows for active rebalancing. As for other applications some have
alluded to, this constraint is only for the routing gossip. Should there
be a good reason to allow increasing the amount forwarded by a peer,
e.g., node n receives x from the previous hop and forwards x+e to the
next hop, that can still be negotiated out of band or even in the onion
payload for that node.

Cheers,
Christian
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] negative fees for HTLC relay

[Mark Friedenbach]

Negative fees also come up in the context of peer to peer credit using self 
issued IOUs (over colored coins or whatever) that are atomically swapped via a 
lightning HTLC. In this case negative fees may be the norm as there is 
incentive to rebalance from higher to lower interest IOUs.

> On Jan 16, 2018, at 2:45 PM, Will Yager  wrote:
> 
> I agree. Negative shadow prices are incredibly important for optimality of 
> constrained network markets where flows in opposite directions cancel (as is 
> the case with lightning). See for example FTRs.  It’s unclear to me how well 
> the analogy holds, but it’s worth considering. 
> 
> —Will
> 
>> On Tue, Jan 16, 2018 at 3:32 PM, Benjamin Mord  wrote:
>> 
>> Thanks. It sounds like it was dropped due to difficulty in the routing 
>> protocol. Is that difficulty documented somewhere I can review? If so, I 
>> might take a crack at a solution to it. But regardless I suggest the 
>> protocol should support negative fees, even if an individual routing 
>> implementation prefers to treat as 0 for simplicity. That should be up to 
>> the implementation I think, and not a protocol constraint.
> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] negative fees for HTLC relay

[Will Yager]

I agree. Negative shadow prices are incredibly important for optimality of 
constrained network markets where flows in opposite directions cancel (as is 
the case with lightning). See for example FTRs.  It’s unclear to me how well 
the analogy holds, but it’s worth considering.

—Will

On Tue, Jan 16, 2018 at 3:32 PM, Benjamin Mord  wrote:

> Thanks. It sounds like it was dropped due to difficulty in the routing 
> protocol. Is that difficulty documented somewhere I can review? If so, I 
> might take a crack at a solution to it. But regardless I suggest the protocol 
> should support negative fees, even if an individual routing implementation 
> prefers to treat as 0 for simplicity. That should be up to the implementation 
> I think, and not a protocol constraint.___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] negative fees for HTLC relay

[Benjamin Mord]

Thanks. It sounds like it was dropped due to difficulty in the routing
protocol. Is that difficulty documented somewhere I can review? If so, I
might take a crack at a solution to it. But regardless I suggest the
protocol should support negative fees, even if an individual routing
implementation prefers to treat as 0 for simplicity. That should be up to
the implementation I think, and not a protocol constraint.

On Tue, Jan 16, 2018 at 2:58 PM, William Casarin  wrote:

> Benjamin Mord  writes:
> > [..]
> > why not allow negative fees to incent unwinding, in scenarios where nodes
> > consider that cheaper than on-chain rebalancing?
>
> This was brought up before here [1]:
>
> Rusty Russell  writes:
> >> Edward Marynarz  writes:
> >> Another trivial question: can the fee be negative? It might help with
> some
> >> channel rebalancing.
>
> >In my original implementation, they could be.  However, that turns out
> >to be a very strange idea, and complicates routing.
>
> [1] https://lists.linuxfoundation.org/pipermail/lightning-dev/
> 2017-December/000827.html
>
> Cheers,
>
> --
> https://jb55.com
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] negative fees for HTLC relay

[William Casarin]

Benjamin Mord  writes:
> [..]
> why not allow negative fees to incent unwinding, in scenarios where nodes
> consider that cheaper than on-chain rebalancing?

This was brought up before here [1]:

Rusty Russell  writes:
>> Edward Marynarz  writes:
>> Another trivial question: can the fee be negative? It might help with some
>> channel rebalancing.

>In my original implementation, they could be.  However, that turns out
>to be a very strange idea, and complicates routing.

[1] 
https://lists.linuxfoundation.org/pipermail/lightning-dev/2017-December/000827.html

Cheers,

-- 
https://jb55.com
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev