Re: [Lightning-dev] Free Rebalancing Proposals

2018-08-13 Thread ZmnSCPxj via Lightning-dev
Good morning Robert,

> Good Morning ZmnSCPxj!
>
> I was thinking using the normal onion-routing, probably modified in some way 
> so it can read and modify max. Must admit I haven't studied that part at that 
> level at all.
>
> For simple three-member constructs it could be enough with a simple non-onion 
> message that A asks B if it would like to rebalance AB,BC and A asks C if it 
> would like to balance BC,CA.
>
> But first it seems I have to at least try to convince you that rebalancing 
> does help solve some problems.
>
> I think rebalancing is good for many reasons.
> Consider a node A that has the following channel balances. It initiated A-B 
> with 100sat and used that up instantly. C initiated an inbound channel of 
> 200sat. A made a new channel A-D with 300sat
> It ended up looking like the following:
>
> A-B 0-100
> A-C 0-200
> A-D 300-0
>
> For simplicity in this case the total balance is 300-300. Also lets say 
> "balanced" means "able to send and receive 25 sat". Achieving total balance 
> on all channels is not realistic. It's up to each node to decide their own 
> definition.
>
> A has only one channel to send out via. If D goes down it has even fewer.
> So my idea is: A finds if there exists any D-C channel (or multi-hop), and 
> politely asks whomever it may concern if sending 100 sat A->D->C->A would 
> benefit them as well. if it does you send over 100 sat and will end up with
>
> A-B 0-100
> A-C 100-100
> A-D 200-100

But what is the state before and the state after of channel D-C?  Have you 
considered?

I postulate that in any case where rebalancing is possible, then a payment 
route exists that is sufficient for payment to the network, and in the end, the 
purpose of LN is payment, not some ideal of channel balance.  This idea is a 
consequence of studying what I call "cyclic superhubs" where cycles of channels 
start with each channel consistently unbalanced in one direction and yet every 
node on that cycle is capable of paying to every other node on that cycle 
without need of explicit rebalancing.  Hence my reluctance to consider the 
addition of rebalancing at all.

I think dynamic fee changing (lowering and increasing fees for channel 
directions you wish to rebalance away/towards you) is sufficient, and requires 
no update to the base LN protocol (but requires updating of actual 
implementations, since they all use fixed fees).

Regards,
ZmnSCPxj___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] Free Rebalancing Proposals

2018-08-13 Thread Robert Olsson
Good Morning ZmnSCPxj!

I was thinking using the normal onion-routing, probably modified in some
way so it can read and modify max. Must admit I haven't studied that part
at that level at all.

For simple three-member constructs it could be enough with a simple
non-onion message that A asks B if it would like to rebalance AB,BC and A
asks C if it would like to balance BC,CA.

But first it seems I have to at least try to convince you that rebalancing
does help solve some problems.

I think rebalancing is good for many reasons.
Consider a node A that has the following channel balances. It initiated A-B
with 100sat and used that up instantly. C initiated an inbound channel of
200sat. A made a new channel A-D with 300sat
It ended up looking like the following:

A-B 0-100
A-C 0-200
A-D 300-0

For simplicity in this case the total balance is 300-300. Also lets say
"balanced" means "able to send and receive 25 sat". Achieving total balance
on all channels is not realistic. It's up to each node to decide their own
definition.

A has only one channel to send out via. If D goes down it has even fewer.
So my idea is: A finds if there exists any D-C channel (or multi-hop), and
politely asks whomever it may concern if sending 100 sat A->D->C->A would
benefit them as well. if it does you send over 100 sat and will end up with

A-B 0-100
A-C 100-100
A-D 200-100

If the agreed max along the path is lower, it will have to do, it helped a
little bit, but lets say 100sat was ok with everyone
Repeat for A->D->B->A and 50sat and you will have

A-B 50-50
A-C 100-100
A-D 150-150

Now A instantly have 3 times as many possible routes out, and 50% more
possible routes in, nobody had to go to the chain to close/open/splice, and
nobodys channel went unbalanced, and nobody paid anyone.

The possibilites for others in the network to actually route via A
increases to all 6 paths from 2, before only B->A->D and C->A->D would
succeed. So there is also a 3-fold increase.

Consider a bigger node with say 40 channels where 10 has too low local
balance and 30 has too low remote balance. It would have 300 working paths
out of 1560, so about 80% failure rate.

So, for redundancy and maximum possible routes, I think rebalancing is good.

The main difference with this protocol compared to how we balance lightning
channels today is

* that everyone benefits from it (or else would deny to be a part of it) so
there are not any fees
* today channel balances sometimes start oscillating if two nodes try to
balance a channel somewhere in the network according to their own will,
without caring/knowing about the intermediates wishes or eachothers. now we
care.
* less trial and error since you know the perfect amount to send

---

In the case you describe, it is a feature, not a bug.
If you find a path A->B->C->D
C would like to balance BC against CD so it offers a rebalance and accepts
fee-free transactions for things coming in BC and goes out CD. But maybe DB
channel doesn't exist or have too little capacity.
It's wishes to rebalance the channel remains. Then suddenly your payment
goes thru and achieves the same goal, I don't see a problem.
if C did *not* want to balance in that direction, it would have rejected
passing the payment for free
if two payments pass thru the same channels, the second one will be
rejected as normal with "feeinsufficient"

If we do want to avoid this (maybe because too many flood the network
probing to find a free hop), you can send the future payment hash, or part
of it, in the proposal message, and whitelist that hash for 1337 seconds.
Even if whitelisted, when/if the payment packet arrives, the node shall
check in channel and amount to see if it is still beneficial, otherwise
just reject it. The agreed max is a soft promise.

Regarding receive-only or send-only nodes and on/off swapping, I think
those are entirely separate problems. My solution only aims at spreading
the existing grease around the layer two for smoother operation.

Best Regards
Robert Olsson





On Mon, Aug 13, 2018 at 1:16 PM, ZmnSCPxj  wrote:

> Good morning Robert,
>
> I assume, this functionality is transported also via the same
> onion-routing-packet style that current payment routing uses?
>
> If so, then it seems this feature can be used by a payer and payee to
> coordinate to locate some route that would, if paid normally, charge fees,
> but if using the "free rebalancing" could be made free.  The payer simply
> locates some route, adds a synthetic initial balance that is much higher
> than the payment to be forwarded, and then request for a "free rebalancing"
> from the payer to the payee.  If at the payee node, the resulting "max" is
> greater than the actual payment being sent, then the payer and payee have
> found a fee-free route between them.
>
> After all, since the functionality is transported by onion-routing-packet,
> then intermediate hop routes cannot know that the ultimate source and
> ultimate 

Re: [Lightning-dev] Free Rebalancing Proposals

2018-08-13 Thread ZmnSCPxj via Lightning-dev
Good morning Robert,

I assume, this functionality is transported also via the same 
onion-routing-packet style that current payment routing uses?

If so, then it seems this feature can be used by a payer and payee to 
coordinate to locate some route that would, if paid normally, charge fees, but 
if using the "free rebalancing" could be made free.  The payer simply locates 
some route, adds a synthetic initial balance that is much higher than the 
payment to be forwarded, and then request for a "free rebalancing" from the 
payer to the payee.  If at the payee node, the resulting "max" is greater than 
the actual payment being sent, then the payer and payee have found a fee-free 
route between them.

After all, since the functionality is transported by onion-routing-packet, then 
intermediate hop routes cannot know that the ultimate source and ultimate 
destination are different nodes.

Note that intermediate nodes cannot detect this at all.  A fee-free rebalance 
request that follows the fee-free rebalance query may be lower than the max it 
forwarded, since, a later intermediate node on the route might have lowered the 
maximum already.

--

For myself, I think rebalancing is not a particular good solution to anything 
(similar to splice).

Imagine a simple 3-member cyclic-superhub network composed of A -> B -> C 
(looping back to -> A).  Node A may pay node C via B, and so on.  Now if C is a 
popular merchant for both B and A, then both the B-C and C-A channels will 
saturate in favor of C.  In such a case, no rebalancing is possible.

A better solution is for C to purchase more incoming balance by asking either 
of B or A to transfer onchain funds in exchange for a preimage (i.e. swap 
offchain funds for onchain funds).  This is effectively "batching" the earnings 
that C gains in LN: C might earn a lot of small amounts, each one too small 
individually to justify onchain transfers, but which batched together can 
justify the onchain transfer.  Further, this batching is noncustodial and C can 
close the channel at any time (incurring an onchain fee), unlike current 
payment processors (PayPal, credit cards) where payment batching is custodial 
and may be frozen by the custodian; on LN, at worst the money will be locked 
for a few days or weeks, but will definitely be returned to you as long as you 
did not actually cheat.

Further this swapping can be done on a separate layer/software that interacts 
with ordinary existing LN software without requiring additional changes at the 
LN protocol level, which is always a good thing: I believe we should prefer to 
make LN a layer that supports further layers, not put every possible feature 
into LN.

This off-to-on and on-to-off swapping solves the same problems that both 
rebalancing and splcing do (channel balancing, incoming channels, channel 
reloading) without requiring changes at the LN layer, merely adding an 
application on top of LN.

Regards,
ZmnSCPxj

Sent with [ProtonMail](https://protonmail.com) Secure Email.

‐‐‐ Original Message ‐‐‐
On August 12, 2018 7:31 AM, Robert Olsson  wrote:

> Dear All,
>
> I'd like to suggest a new function in the network, "Free Rebalancing 
> Proposals"
> It is a fee-less, omni-beneficial rebalancing scheme
>
> Background:
>
> I currently balance my nodes in the following way, extremely simplified:
>
> 1) Identify channels that have too low remote_balance (A) and too low 
> local_balance (B) for being of any use (in that direction)
> 2) Find potential routes and amounts between all channels A and all channels 
> B that would optimize at least one of them
> 3) Send a payment along those possible routes for different amounts, sorted 
> after fee, until one succeeds
>
> It works quite good and is quite straight forward until the third step. So I 
> would like to propose a better solution which works like this:
>
> Send a proposal-for-balancing-message over channel A, and the max amount you 
> would propose to balance in that direction.
>
> Each node analyses the in and out channels and adjusts the max to it's own 
> desire to balance. Max can only be adjusted down. Let's say the function is 
> `int freeBalancerMax(channel in,channel out)`. It could be as simple as 
> checking how near "in" is to 100ksat local_balance and "out" is to 100ksat 
> remote_balance. The node passes that message with new max value on to the 
> next node, much like a normal payment.
>
> When this packet comes back to the origin it will know how much the 
> intermediate nodes are willing to let you balance, or if someone is reluctant 
> to balance (and the final max value would be 0)
>
> If everyone is happy (i.e. max>0), the origin node sends a payment along this 
> node, *without* dropping off any fees along the way, since all nodes agreed 
> and benefit from the rebalancing act.
>
> Since all intermediate nodes benefit they are happy to do this without fees. 
> Since they adjust max values to their desire it will stop multiple attempts 

[Lightning-dev] Free Rebalancing Proposals

2018-08-11 Thread Robert Olsson
Dear All,

I'd like to suggest a new function in the network, "Free Rebalancing
Proposals"
It is a fee-less, omni-beneficial rebalancing scheme

Background:

I currently balance my nodes in the following way, extremely simplified:

1) Identify channels that have too low remote_balance (A) and too low
local_balance (B) for being of any use (in that direction)
2) Find potential routes and amounts between all channels A and all
channels B that would optimize at least one of them
3) Send a payment along those possible routes for different amounts, sorted
after fee, until one succeeds

It works quite good and is quite straight forward until the third step. So
I would like to propose a better solution which works like this:

Send a proposal-for-balancing-message over channel A, and the max amount
you would propose to balance in that direction.

Each node analyses the in and out channels and adjusts the max to it's own
desire to balance. Max can only be adjusted down. Let's say the function is
`int freeBalancerMax(channel in,channel out)`. It could be as simple as
checking how near "in" is to 100ksat local_balance and "out" is to 100ksat
remote_balance. The node passes that message with new max value on to the
next node, much like a normal payment.

When this packet comes back to the origin it will know how much the
intermediate nodes are willing to let you balance, or if someone is
reluctant to balance (and the final max value would be 0)

If everyone is happy (i.e. max>0), the origin node sends a payment along
this node, *without* dropping off any fees along the way, since all nodes
agreed and benefit from the rebalancing act.

Since all intermediate nodes benefit they are happy to do this without
fees. Since they adjust max values to their desire it will stop multiple
attempts from different nodes from starting to oscillate the balances in
the channels, but will instead reach some kind of equilibrium as long as
the node suggesting proposal uses it's freeBalancerMax when proposing and
accepting proposals.

The intermediate nodes might not need to keep track of any agreements to
balancing proposals, it can simply call the freeBalancerMax function, and
if the payment is lower, it passes on the payment even if it didn't get any
fees, otherwise it fails as usual. If someone tries to send a non agreed
payment without fees, but it coincides with the nodes own desire to
rebalance, there is no harm done. Only problem with that is that senders
might flood a path with creative attempts trying to find a path with a
fee-free hop. I think things like that will be stopped by a general
throttle functionality which probably will be needed anyhow.

Best regards
Robert Olsson
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev