Re: [Lightning-dev] type,len,value standard

2018-11-14 Thread ZmnSCPxj via Lightning-dev
Good morning Conner,

> > For a sequence of `type,len,value`, the `type`s must be in ascending order
> > -- not explicitly accepted or rejected. It would be easier to check
> > uniqueness > (the previous rule we accepted) here for a naive parser (keep
> > track of some "minimum allowed type" that initializes at zero, check current
> > type >= this, update to current type + 1) if `type`s are in ascending order.
>
> Yep ascending makes sense to me, for the reasons you stated.

We could bikeshed this and point out that descending would make just as much 
sense and would work just as well, especially if we want to waste precious 
electrons on debating this.

>
> > 1, `type` - one byte or two?
>
> I'd lean towards one, if a message has 256 optional fields, it might be time 
> to
> consider a new message type altogether.

This rationale seems sensible.

>
> > 3.  `type` - does "it's OK to be odd" apply? i.e. if an even `type` that is
> > not known is found, crash and burn. But intent of this system is for 
> > future
> > expansion for optional fields, so...?
> >
>
> Perhaps this depends on context:
>
> -   for gossip messages, I think the primary concern is not breaking signature
> validation, and that these would need to remain optional for backwards
> compatibility.
>
> -   for link-level messages, we have a little more control. I imagined the 
> fields
> would be gated by feature bit negotiation, and deviating from
> unsupported/required would result in being disconnected.

Then I suppose gossip messages could always just use odd type/key, there would 
still be 128 of those with a single byte, and at least we would have some sort 
of consistency, which could help if our software factors out some consistency 
checks for all t-l-v.

>
>
> > 5.  `len` - one byte or two? I believe we tend to use two bytes for various
> > lengths.
> >
>
> Maybe varint? One byte is not enough for all lengths, but two seems excessive
> for uint8 or even uint32.

Given that messages are currently only up to 65536 bytes total, is that not a 
bit much?

>
> > 6.  BOLT - I propose making a separate BOLT for `type,len,value`, which 
> > other
> > messages and so on simply refer to.
> >
>
> Indeed, are you thinking we'd use this to add new fields proposed in 1.1?

Yes.
It would also be good to have a single place to describe the new scheme.
I also imagine that we would do the sensible thing and create a new 
type/module/class/whatever for t-l-v data structures (a mapping from u16 to 
binary blobs).
Having a separate BOLT document would help us spec and verify this in a 
reasonably separate manner.

>
> In addition to the above, do we also want to flesh out what sub-TLV structures
> would look like? Or perhaps that isn't necessary, if we can continue adding 
> more
> root-level keys.

No, I imagine at the t-l-v level we see only binary blobs for the value, and 
separate parts of our software would understand what the actual meanings of 
those blobs are.

Regards,
ZmnSCPxj
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] Link-level payment splitting via intermediary rendezvous nodes

2018-11-14 Thread Christian Decker
ZmnSCPxj  writes:
> The construction we came up with allows multiple rendezvous nodes,
> unlike the HORNET construction that is inherently only a single
> rendezvous node.  Perhaps the extra flexibility comes with some
> security degradation?

I don't think this is the case. If I remember correctly (Conner please
correct me if I'm wrong here), then the Hornet rendez-vous construction
relied on a Sphinx packet from the RV to R, wrapped in a Sphinx packet
from S to RV. This was possible because of the variable sized payload.
It would be possible to do that a number of times, with the downside
that the packet would be bigger and bigger since we are wrapping full
Sphinx packets.

Our construction with the ephemeral key switch at the rendez-vous point
is identical to that construction, except that we have the ephemeral key
at the RV hidden inside the routing information (per-hop payload) and
the remainder of the route in what would otherwise be padding. The
constructions are IMHO no different except for the location we store the
forward information that the RV will have to unpack (per-hop payload
instead of nested sphinx packets).

The only difficulty that I pointed out comes from the fact that the HMAC
verification can't work if we can't generate a specify shared secret at
the RV, which to me sounds like an intrinsic property of the way we use
one-way functions to derive those.

Cheers,
Christian
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] type,len,value standard

2018-11-14 Thread Conner Fromknecht
Hi ZmnSCPxj,

Thanks for writing this up! I had started an email, but you beat me to it :)

> 1.  For a sequence of `type,len,value`, each `type` must be unique. --
> accepted.

To add to this, it seemed that there was some agreement that repeated fields
should be serialized under a single root key, since a receiver can't know if a
field is allowed to have duplicates if they don't understand the field.

> For a sequence of `type,len,value`, the `type`s must be in ascending order
> -- not explicitly accepted or rejected.  It would be easier to check
> uniqueness > (the previous rule we accepted) here for a naive parser (keep
> track of some "minimum allowed type" that initializes at zero, check current
> type >= this, update to current type + 1) if `type`s are in ascending order.

Yep ascending makes sense to me, for the reasons you stated.

> 1, `type` - one byte or two?

I'd lean towards one, if a message has 256 optional fields, it might be time to
consider a new message type altogether.

> 3. `type` - does "it's OK to be odd" apply?  i.e. if an even `type` that is
> not known is found, crash and burn.  But intent of this system is for future
> expansion for optional fields, so...?

Perhaps this depends on context:
 - for gossip messages, I think the primary concern is not breaking signature
 validation, and that these would need to remain optional for backwards
 compatibility.
 - for link-level messages, we have a little more control. I imagined the fields
   would be gated by feature bit negotiation, and deviating from
   unsupported/required would result in being disconnected.

> 5. `len` - one byte or two? I believe we tend to use two bytes for various
> lengths.

Maybe varint? One byte is not enough for all lengths, but two seems excessive
for uint8 or even uint32.

> 6.  BOLT - I propose making a separate BOLT for `type,len,value`, which other
> messages and so on simply refer to.

Indeed, are you thinking we'd use this to add new fields proposed in 1.1?

In addition to the above, do we also want to flesh out what sub-TLV structures
would look like? Or perhaps that isn't necessary, if we can continue adding more
root-level keys.

--Conner
On Wed, Nov 14, 2018 at 8:54 PM ZmnSCPxj via Lightning-dev
 wrote:
>
> Good morning list,
>
> An item added discussed in the summit was the proposed "type,len,value", 
> which is added to the end of messages and other intercommunication structures 
> (invoices and so on).
> This would allow some transition to future additional fields while 
> maintaining backward compatibility.
>
> I believe these were brought up:
>
> 1.  For a sequence of `type,len,value`, each `type` must be unique. -- 
> accepted.
> 2.  For a sequence of `type,len,value`, the `type`s must be in ascending 
> order -- not explicitly accepted or rejected.  It would be easier to check 
> uniqueness (the previous rule we accepted) here for a naive parser (keep 
> track of some "minimum allowed type" that initializes at zero, check current 
> type >= this, update to current type + 1) if `type`s are in ascending order.
>
> Now for bikeshedding:
>
> 1, `type` - one byte or two?
> 2. `type` - maybe some other name, since we already use `type` for messages?  
> How about, `key` instead?
> 3. `type` - does "it's OK to be odd" apply?  i.e. if an even `type` that is 
> not known is found, crash and burn.  But intent of this system is for future 
> expansion for optional fields, so...?
> 4. `len` - measures bytes of `value`, obviously since if the receiver does 
> not know the `type` then it cannot know what unit is used for the `value`.
> 5. `len` - one byte or two? I believe we tend to use two bytes for various 
> lengths.
> 6.  BOLT - I propose making a separate BOLT for `type,len,value`, which other 
> messages and so on simply refer to.
>
> Regards,
> ZmnSCPxj
>
> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] Strawman BOLT11 static "offer" format using probes.

2018-11-14 Thread ZmnSCPxj via Lightning-dev
Good morning Rusty,

No particular comment on static offer invoices, but instead various 
bikeshedding.

>
> The format of the final-hop lightning onion would contain:
>
> [whatever-marker-we-need?][128-bit-`p`-field][[type,len,data]+]

I think a separate BOLT for the type,len,data would be useful, and might also 
document various consistent designs of messages (such as var-length fields 
using a prefixed 16-bit length measuring number of items in the field).

BOLT #13: type,len,data standard

(I should probably move this to a new thread).

> I apologize that this wasn't fleshed out before the summit, but I
> overestimated the power of Scriptless Scripts so had mentally deferred
> this.

My understanding is that SS *is* as powerful as we thought, at least for some 
of the applications we were hoping to use it for.
However, implementing SS is hard without Schnorr, because script magic with 
`OP_CODESEPARATOR` is magic, and we essentially stalled out and said "maybe 
wait for Schnorr instead".


Regards,
ZmnSCPxj
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


[Lightning-dev] Strawman BOLT11 static "offer" format using probes.

2018-11-14 Thread Rusty Russell
Hi all,

I want to propose a method of having reusable BOLT11 "offers" which
provide almost-spontaneous payments as well as not requiring generating
a BOLT11 invoice for each potential sale.

An "offer" has a `p` field of 26 bytes (128 bits assuming top two are 0)
(which is ignored by existing nodes).  The payer uses a new lightning
probe message using the current onion format we use for HTLCs to
retreive the complete invoice.

The format of the final-hop lightning onion would contain:

[whatever-marker-we-need?][128-bit-`p`-field][[type,len,data]+]

We would probably define a few optional types to start:

1. quantity: for ordering multiple of an item, default 1.
2. delivery-address: steal from 
https://www.w3.org/TR/vcard-rdf/#Delivery_Addressing_Properties ?
3. signature: basically a blob so payer can prove it was them.

The return lightning message would contain a new bolt11 invoice (perhaps
we optimize some fields by copying from the bolt11 offer if they don't
appear?), and an additional field:

`m` (27) `data_length` 52.  Merkle hash of fields payer provided
in onion msg above, and the offer `p` value.

The payer checks the signature is correct, `m` is correct, and uses the
invoice to pay as normal.  The bolt11 offer + fields-from-onion + bolt11
invoice + preimage is the complete proof of payment.

Refinements
---

We can generate alternate leaves for the merkle tree (using
SHA256(shared-secret | leafnum)) so revealing the `m` value doesn't risk
revealing your delivery-address for example.

The return needs to list the fields it *didn't* include in the merkle
because it didn't accept them (the merchant doesn't want to be bound to
conditions it doesn't understand!).

We could add a `k` field to the bolt11 offer to allow the final invoice
to delegated to a separate key.

The default `x` (expiry) field for an offer which does not have an
old-style 53-byte `p` field (ie. a "pure" offer) could be infinite.

We could merkelize the delivery-address too :)

I've handwaved a bit over the detailed format, because there are other
things we want to put in the onion padding, and because the return is
similar to the "soft-error"/"partial payment ack" proposals.

Results
---

This gives us static invoicing, and a single static invoice (without an
amount field) can thus be used to approximate "spontaneous" donations,
while still providing proof of payment; indeed, providing
non-transferrable proof-of-payment since the invoice now commits to the
payer-provided signature.

It also provides a platform for recurring payments: while we can do this
with preimage-is-next-payment_hash, that requires pre-generation and
isn't compatible with static invoices.

I apologize that this wasn't fleshed out before the summit, but I
overestimated the power of Scriptless Scripts so had mentally deferred
this.

Thanks!
Rusty.
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] Base AMP

2018-11-14 Thread ZmnSCPxj via Lightning-dev
Good morning list,

In case it was not noticed, I made a pull request for Base AMP: 
https://github.com/lightningnetwork/lightning-rfc/pull/511

This is primarily based on what Rusty suggested on-list, with sufficient MUST 
and SHOULD.

Regards,
ZmnSCPxj


Sent with ProtonMail Secure Email.

‐‐‐ Original Message ‐‐‐
On Wednesday, November 14, 2018 9:59 AM, ZmnSCPxj via Lightning-dev 
 wrote:

> Good morning Rusty,
>
> Someone pointed out to me that `intended_payment_amount` is unnecessary.
> On reflection, this is correct.
> Both intermediate nodes and the payee node need not have 
> `intended_payment_amount`.
>
> Therefore
>
> > > I propose the below to support Base AMP.
> >
> > I think the complexity outweighs the benefits for the moment. The
> > sender would have to make the onions identical past the merge point (so
> > any one of them could be used), the merge point has to now create a
> > many:1 map for HTLC redemption.
> > For the moment, I think we should stick with:
> > BOLT #4:
> >
> > 1.  type: `per_hop`
> > 2.  data:
> > -   [`8`:`short_channel_id`]
> > -   [`8`:`amt_to_forward`]
> > -   [`4`:`outgoing_cltv_value`]
> >
> > -   -   [`12`:`padding`]
> > -   -   [`1`:`flags`]
> > -   -   [`11`:`padding`]
> > And define bit 0 of `flags` as `incomplete_payment`. For the 
> > moment, it
> > is only allowed for final nodes, and only if they put it in their 
> > BOLT11
> > field.
> >
>
> We can do something even simpler.
>
> If `amt_to_forward` plus the fees charged by this node is greater than the 
> actual incoming HTLC, this is an AMP attempt.
> No additional flag needs to be added.
> For final payment nodes, if the `amt_to_forward` is greater than the incoming 
> HTLC value, this is an AMP attempt.
>
> The previous node could try to probe this by offering a smaller amount than 
> it was instructed to give, but cannot differentiate between a stall because 
> the payee is waiting for an AMP, or a stall due to some other unrelated error.
>
> --
>
> Of course, an explicit flag is more sensible as it is more explicit.
>
> For myself, I would rather a new `per_hop_type`, but whether to use a 
> separate `per_hop_type` vs a byte in padding is largely a bikeshed issue and 
> either is fine with me.
> A concern is that nothing in our current spec requires that `padding` be all 
> 0, such that reinterpreting byte 0 to be flags could cause interoperability 
> problems.
> So perhaps a new `per_hop_type` which has a 2-byte `flags` (for more future 
> expansion) and a `padding` of 10 bytes which MUST be explicitly specced as 
> "MUST be all 0".
>
> An explicit flags field would also allow delivery of higher-layer application 
> data in each payment, for whatever purpose a higher-layer application may 
> want. E.g. bit 1 could mean "the next hop 65 bytes is actually a 32-byte 
> application ID and a 33-byte payload; this flag is valid only if this is the 
> last hop."
> Another bit can also be used to provide spontaneous payment, so e.g. bit 2 
> could mean "this hop is the final hop (even if HMAC is nonzero); the HMAC of 
> this hop is really the preimage to claim this payment."
>
> Regards,
> ZmnSCPxj
>
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] Summary of the Second Lightning Development Summit (2018 Adelaide)

2018-11-14 Thread ZmnSCPxj via Lightning-dev
While consistency with typical grammar might be desirable, as a made-up word, 
wumbo has the opportunity to define its own grammar.
Do we really want to impose normal grammar on wumbo?
We do not want to paint ourselves in a corner if in the future we find we need 
the ability to define novel grammar for made-up words.

Granted, we probably should not "roll our own grammar" without some proof of 
understandability.


Sent with ProtonMail Secure Email.

‐‐‐ Original Message ‐‐‐
On Wednesday, November 14, 2018 10:00 PM, Pierre  wrote:

> > I wumbo, you wumbo, he wumbo she wumbo.
>
> I would respectfully disagree here: pretty sure that's "he
> wumboes, she wumboes".


___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] Summary of the Second Lightning Development Summit (2018 Adelaide)

2018-11-14 Thread ZmnSCPxj via Lightning-dev
The link in the original post contains actual descriptions of "wumbo" and 
"hidden destinations".
Admittedly, the linked wiki page is not necessarily clear, so I shall describe 
them here anyway.

Wumbo is the study of all things wumbological.
I wumbo, you wumbo, he wumbo she wumbo.
Wumboing.
Wumbological, being or having wumbo.
If "M" is "mini", we can turn around the "M" into "W", which means "wumbo".
In short, more clearly, "wumbo" mans "maxi" or removing the channel capacity 
limit.
If both sides of a new channel agree to wumbo each other by setting 
`option_i_wumbo_you_wumbo`, they can build channels with capacity higher than 
167.77216mBTC.
A node that advertises `option_wumborama` allows any node to build channels 
with capacity above the limit.
Please blame one of the persons attending the summit for this term.
Unfortunately, due to rules imposed, non-attendees can only learn who to blame 
if that person admits this.

Hidden destinations is basically "rendezvous routing".
See "Rendez-vous Routing" thread by CJP on the list, and the discussion on 
thread "Link-level payment splitting via intermediary rendezvous nodes" on the 
list.

Regards,
ZmnSCPxj

Sent with [ProtonMail](https://protonmail.com) Secure Email.

‐‐‐ Original Message ‐‐‐
On Wednesday, November 14, 2018 8:41 PM, Cezary Dziemian 
 wrote:

> Thanks,
>
> Great work. I'am excited especially by dual-funded channels.
>
> Unfortunately I don't know what  "Wumbo" and "hidden destinations" are. Could 
> you explain or provide some links?
>
> What about eltoo?
>
> Best Regrads,
> Cezary Dziemian
>
> wt., 13 lis 2018 o 00:29 Rusty Russell  napisał(a):
>
>> Two years ago, a dozen Lightning implementers met in Milan to start the
>> Lightning Specification 1.0.  Last week the 18 most active spec
>> contributors gathered in Adelaide to kick off version 1.1.
>>
>> After two days we emerged with 30 accepted changes for next version,
>> many very detailed and some requiring further on-list refinement.
>> Highlights include multi-path payments, dual-funded channels, splicing,
>> wumbo, hidden destinations, and many gossip improvements.
>>
>> Please join in the continuing discussion: the work points have been
>> transferred to the Wiki:
>>
>> 
>> https://github.com/lightningnetwork/lightning-rfc/wiki/Lightning-Specification-1.1-Proposal-States
>>
>> Thanks to all those who attended and patiently contributed their
>> expertise to some extremely difficult questions; it was an honor to be
>> in the room with you all.
>>
>> Cheers,
>> Rusty.
>> ___
>> Lightning-dev mailing list
>> Lightning-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] Link-level payment splitting via intermediary rendezvous nodes

2018-11-14 Thread Christian Decker
Hi Conner,

thanks for the pointers, looking forward to reading up on the
wrap resistance. I don't quite follow if you're against the
re-wrapping for spontaneous re-routing, or the entire rendez-vous
construction we came up with in Australia. If it's the latter, do
you have an alternative construction that we might look at?
Hornet requires the onion-in-onion initial sphinx setup IIRC
which is pretty much what we came up with here (with the
exception that we manage to have the second onion be hidden in
the first one's header instead of the payload).

Cheers,
Christian

On Tue, Nov 13, 2018 at 9:21 PM Conner Fromknecht
 wrote:

> Good morning all,
>
> Taking a step back—even if key switching can be done mathematically, it
> seems
> dubious that we would want to introduce re-routing or rendezvous routing
> in this
> manner. If the example provided _could_ be done, it would directly violate
> the
> wrap-resistance property of the ideal onion routing scheme defined in [1].
> This
> property is proven for Sphinx in section 4.3 of [2]. Schemes like HORNET
> [3]
> support rendezvous routing and are formally proven in this model. Seems
> this
> would be the obvious path forward, given that we've already done a
> considerable
> amount of work towards implementing HORNET via Sphinx.
>
> Cheers,
> Conner
>
> [1] A Formal Treatment of Onion Routing:
> https://www.iacr.org/cryptodb/archive/2005/CRYPTO/1091/1091.pdf
> [2] Sphinx: https://cypherpunks.ca/~iang/pubs/Sphinx_Oakland09.pdf
> [3] HORNET: https://arxiv.org/pdf/1507.05724.pdf
> On Mon, Nov 12, 2018 at 8:47 PM ZmnSCPxj via Lightning-dev
>  wrote:
> >
> > Good morning Christian,
> >
> > I am nowhere near a mathematician, thus, cannot countercheck your
> expertise here (and cannot give a counterproposal thusly).
> >
> > But I want to point out the below scenarios:
> >
> > 1.  C is the payer.  He is in contact with an unknown payee (who in
> reality is E).  E provides the onion-wrapped route D->E with ephemeral key
> and other data necessary, as well as informing C that D is the rendez-vous
> point.  Then C creates a route from itself to D (via channel C->D or via
> C->A->D).
> >
> > 2.  B is the payer.  He knows the entire route B->C->D->E and knows that
> payee is C.  Unfortunately the C<->D channel is low capacity or down or etc
> etc.  At C, B has provided the onion-wrapped route D->E with ephemeral key
> and other data necessary, as well as informing to C that D is the next
> node.  Then C either pays via C->D or via C->A->D.
> >
> > Even if there is an off-by-one error in our thinking about rendez-vous
> nodes, could it not be compensated also by an off-by-one in the link-level
> payment splitting via intermediary rendez-vous node?
> > In short, D is the one that switches keys instead of A.
> >
> > The operation of processing a hop would be:
> >
> > 1.  Unwrap the onion with current ephemeral key.
> > 2.  Dispatch based on realm byte.
> > 2.1.  If realm byte 0:
> > 2.1.1.  Normal routing behavior, extract HMAC, etc etc
> > 2.2.  If realm byte 2 "switch ephemeral keys":
> > 2.2.1.  Set current ephemeral key to bytes 1 -> 32 of packet.
> > 2.2.2.  Shift onion by one hop packet.
> > 2.2.3.  Goto 1.
> >
> > Would that not work?
> > (I am being naive here, as I am not a mathist and I did not understand
> half what you wrote, sorry)
> >
> > Then at C, we have the onion from D->E, we also know the next ephemeral
> key to use (we can derive it since we would pass it to D anyway).
> > It rightshifts the onion by one, storing the next ephemeral key to the
> new hop it just allocated.
> > Then it encrypts the onion using a new ephemeral key that it will use to
> generate the D<-A<-C part of the onion.
> >
> > Regards,
> > ZmnSCPxj
> >
> >
> > Sent with ProtonMail Secure Email.
> >
> > ‐‐‐ Original Message ‐‐‐
> > On Tuesday, November 13, 2018 11:45 AM, Christian Decker <
> decker.christ...@gmail.com> wrote:
> >
> > > Great proposal ZmnSCPxj, but I think I need to raise a small issue with
> > > it. While writing up the proposal for rendez-vous I came across a
> > > problem with the mechanism I described during the spec meeting: the
> > > padding at the rendez-vous point would usually zero-padded and then
> > > encrypted in one go with the shared secret that was generated from the
> > > previous ephemeral key (i.e., the one before the switch). That
> ephemeral
> > > key is not known to the recipient (barring additional rounds of
> > > communication) so the recipient would be unable to compute the correct
> > > MACs. There are a number of solutions to this, basically setting the
> > > padding to something that the recipient could know when generating its
> > > half onion.
> > >
> > > My current favorite goes like this:
> > >
> > > 1.  Rendez-vous RV receives an onion, performs ECDH like normal to get
> > > the shared secret, decrypts its payload, simultaneously encrypts
> > > the padding.
> > >
> > > 2.  It extracts its per-hop payload and shifts the entire 

[Lightning-dev] Forwarding hints in channel update messages

2018-11-14 Thread Joost Jager
Hello all,

I'd like to bring up an idea that builds on top of "non-strict" forwarding.
I commented about this on conner's non-strict forwarding lightning-rfc pr,
but it is probably better to discuss it on its own in this list.

A node that forwards non-strictly, is using any of its channels to carry
the payment to the next hop. It is ignoring the actually requested channel
in `update_add_htlc`, except for determining the next hop pubkey.

When forwarding fails, a `channel_update` msg can be returned to the
sender. This brings up the question for which channel to return a
`channel_update` in case of failed non-strict forwarding.

If the htlc didn't satisfy the policy of the requested channel, the
sender's view on the graph is not up to date and it makes sense to return a
`channel_update` for the requested channel.

However, if the htlc did satisfy the policy, sending back a
`channel_update` does not provide the sender with any new information. In
case of TemporaryChannelFailure, the update is optional. But leaving it out
does not save any bytes because of padding (as pointed out by pierre in the
pr).

The idea is to repurpose the `channel_update` message in this case as a
'forwarding hint'.

When non-strict forwarding fails, the intermediate node iterates over all
its channels to the next hop and determines what would be the 'best'
channel to use from the sender point of view. Best could be defined as a
trade off between minimum fee and time lock delta, similar to the weight
function used during path finding. Only channels that have enough balance
for the amount requested in the htlc are considered in this process.

If there is no best channel (for example when none of the channels have
enough capacity), the node just returns a `channel_update` for the
requested channel as usual.

If there is a best channel, a `channel_update` is returned for that channel
instead of the requested channel. Senders that are aware of this behavior
will recognize this when reading the failure message and interpret the
`channel_update` as a forwarding hint.

Senders not aware of the forwarding hint will either just apply the channel
update to the graph or reject it. Both are fine, because their copy of the
policy for the requested channel was already up-to-date. This makes this
idea backwards compatible.

What this forwarding hint signals is that an htlc with a fee and time lock
delta that satisfies the policy of the hinted channel will likely be
forwarded successfully. Of course if something changes at the intermediate
node (channel balance) in the mean time, the hint may not be valid anymore.

With the hint in hand, the sender can adjust the route to satisfy the
hinted policy and try again. Alternatively, it could first try a route
through other nodes because the hinted policy increases the total fee
and/or time lock too much. How to exactly integrate this in path finding is
something to work out further. The sender should be aware that an
intermediate node may try to maximize its earning by handing out
'expensive' forwarding hints and therefore should not blindly apply the new
policy to a route.

The advantage of having the hint is that the sender likely needs fewer
payment attempts to complete the payment. For the intermediate node, it is
a way to increase its earnings. It gives the sender more certainty about
the parameters that lead to successful forwarding and the sender may choose
to just go with those instead of trying out many other routes, even if one
of those routes could be better. In case the sender wants the absolute best
route, the forwarding hint may still be beneficial to the intermediate
node. When there are multiple routes with identical total fees and time
locks, a sender will likely choose the route for which it has received
forwarding hints.

In case the intermediate node can only forward the payment over a private
channel, it could hint the policy of a public channel with a policy that
also satisfies the private channel's policy. It doesn't matter if this
public channel doesn't have enough balance, because non-strict forwarding
will also be applied on the next attempt. Or maybe just returning a
`channel_update` for channel id 0 with the private channel's policy.
Senders aware of forwarding hints may just as well interpret this properly.

To implement this, no onion packet changes are required. An intermediate
node could advertise that it provides forwarding hints through a global
feature bit, but that is optional. The forwarding hints can also be
recognized in the `channel_update` message itself.

Regards,
Joost
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev