Re: [Lightning-dev] Base AMP
Except we have invoices with no specified amount (payer dictates how much to pay). Which is why we need to send the total amount to the payee as part of the onion final hop, for the case the invoice has no specified amount. Regards, ZmnSCPxj Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐ Original Message ‐‐‐ On Tuesday, December 4, 2018 7:40 PM, Christian Decker wrote: > Which brings us back to the initial proposal that just signals the awareness > of a temporary underpayment with the single "more is coming"-bit. > > On Sun, Dec 2, 2018 at 11:49 PM Rusty Russell wrote: > >> ZmnSCPxj writes: >>> But what if 2 of those paths fail? >>> It would be better to merge them into a single payment along the expensive >>> 4th path. >>> However, the remaining succeeding path has already given `numpaths`=3. >>> >>> Using `numpaths` overcommits to what you will do in the future, and is >>> unnecessary anyway. >>> The payee is interested in the total value, not the details of the split. >> >> Excellent point. >> >> Thanks, >> Rusty. >> ___ >> Lightning-dev mailing list >> Lightning-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
ZmnSCPxj writes: > But what if 2 of those paths fail? > It would be better to merge them into a single payment along the expensive > 4th path. > However, the remaining succeeding path has already given `numpaths`=3. > > Using `numpaths` overcommits to what you will do in the future, and is > unnecessary anyway. > The payee is interested in the total value, not the details of the split. Excellent point. Thanks, Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning Rusty, Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Friday, November 30, 2018 7:46 AM, Rusty Russell wrote: > ZmnSCPxj zmnsc...@protonmail.com writes: > > > Good morning all, > > > > > I initially suggested we could just have a 2-byte "number of total > > > pieces", but it turns out there's a use-case where that doesn't work > > > well: splitting the bill. There each payer is unrelated, so doesn't > > > know how the others are paying. > > > > This would also not work well in case of a dynamic algorithm that greedily > > tries to pay the whole amount at once, then splits it if it does not fit, > > with each split also being liable to splitting. > > Such a dynamic algorithm would not know in the first place how many splits > > it will take, but it will know the total amount it intends to deliver. > > Well, that would have worked because received takesmax of the values > received, ie, sender starts with A and B, both with "numpieces=2", > then splits B into BA and BB, both with "numpieces=3". Consider a network where there are 4 paths between payer and payee. 3 paths have low capacity but negligible feerate, while the 4th has high capacity but ridiculously high feerates measurable in whole microbitcoins. The rational thing to try, when paying a somewhat large amount but trying to minimize fees, would be to split among the three lowcost paths. But what if 2 of those paths fail? It would be better to merge them into a single payment along the expensive 4th path. However, the remaining succeeding path has already given `numpaths`=3. Using `numpaths` overcommits to what you will do in the future, and is unnecessary anyway. The payee is interested in the total value, not the details of the split. Regards, ZmnSCPxj > > But it's bad for the separate-payer case anyway, so... > > Thanks, > Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
ZmnSCPxj writes: > Good morning all, > >> I initially suggested we could just have a 2-byte "number of total >> pieces", but it turns out there's a use-case where that doesn't work >> well: splitting the bill. There each payer is unrelated, so doesn't >> know how the others are paying. > > This would also not work well in case of a dynamic algorithm that greedily > tries to pay the whole amount at once, then splits it if it does not fit, > with each split also being liable to splitting. > Such a dynamic algorithm would not know in the first place how many splits it > will take, but it *will* know the total amount it intends to deliver. Well, that would have worked because received takes *max* of the values received, ie, sender starts with A and B, both with "numpieces=2", then splits B into BA and BB, both with "numpieces=3". But it's bad for the separate-payer case anyway, so... Thanks, Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning all, > I initially suggested we could just have a 2-byte "number of total > pieces", but it turns out there's a use-case where that doesn't work > well: splitting the bill. There each payer is unrelated, so doesn't > know how the others are paying. This would also not work well in case of a dynamic algorithm that greedily tries to pay the whole amount at once, then splits it if it does not fit, with each split also being liable to splitting. Such a dynamic algorithm would not know in the first place how many splits it will take, but it *will* know the total amount it intends to deliver. Regards, ZmnSCPxj ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Johan Torås Halseth writes: > (excuse me for not yet understanding what this extra complexity gives us) > > To summarize: My suggestion was to only add an optional field to the > invoice, and let the recepient wait until all funds have received before > pulling the payment. No changes to the onion. > > We briefly discussed this during the last call, that the extra bit set in > the onion will be necessary to support Partial Payments (PP?) in the > spontaneous payments case. The donation case: a BOLT11 invoice doesn't have to specify an amount: A writer: ... - If it requires a specific minimum amount for successful payment: - MUST include that `amount` I initially suggested we could just have a 2-byte "number of total pieces", but it turns out there's a use-case where that doesn't work well: splitting the bill. There each payer is unrelated, so doesn't know how the others are paying. I've written up an onion proposal to cover this... Cheers, Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning Johan, I believe what Rusty refers to here is a probe by an intermediate node, rather than a probe by the source node (who, as we know, already knows whether the payee supports AMP or not, by the invoice). Regards, ZmnSCPxj Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐ Original Message ‐‐‐ On Monday, November 26, 2018 3:58 PM, Johan Torås Halseth wrote: > This shouldn't be problem, as the invoice will already indicate that the node > supports BaseAMP. If you have a reason to not reveal that you support BAMP > for certain invoices, you'll just not specify it in the invoice, and act > non-BAMPy when receiving payments to this payment hash. > > Of course, this will also be opt-in for both sides and won't affect existing > nodes in any way. > > Cheers, > Johan > > On Wed, Nov 21, 2018 at 11:54 PM Rusty Russell wrote: > >> Johan Torås Halseth writes: >>> Seems like we can restrict the changes to BOLT11 by having the receiver >>> assume NAMP for incoming payments < invoice_amount. (with some timeout of >>> course, but that would need to be the case even when the sender is >>> signalling NAMP). >> >> This would effectively become a probe for Base AMP; if you get a partial >> payment error, it's because the recipient didn't support Base AMP. >> >> Seems cleaner to have a flag, both on BOLT11 and inside the onion. Then >> it's explicitly opt-in for both sides and doesn't affect existing nodes >> in any way. >> >> Cheers, >> Rusty.___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
This shouldn't be problem, as the invoice will already indicate that the node supports BaseAMP. If you have a reason to not reveal that you support BAMP for certain invoices, you'll just not specify it in the invoice, and act non-BAMPy when receiving payments to this payment hash. Of course, this will also be opt-in for both sides and won't affect existing nodes in any way. Cheers, Johan On Wed, Nov 21, 2018 at 11:54 PM Rusty Russell wrote: > Johan Torås Halseth writes: > > Seems like we can restrict the changes to BOLT11 by having the receiver > > assume NAMP for incoming payments < invoice_amount. (with some timeout of > > course, but that would need to be the case even when the sender is > > signalling NAMP). > > This would effectively become a probe for Base AMP; if you get a partial > payment error, it's because the recipient didn't support Base AMP. > > Seems cleaner to have a flag, both on BOLT11 and inside the onion. Then > it's explicitly opt-in for both sides and doesn't affect existing nodes > in any way. > > Cheers, > Rusty. > ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning Rusty, Okay, I shall modify pull request as you suggested. Regards, ZmnSCPxj Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Thursday, November 22, 2018 6:50 AM, Rusty Russell wrote: > ZmnSCPxj zmnsc...@protonmail.com writes: > > > Good morning Rusty, > > > > > And do not play with `amount_to_forward`, as it's an important > > > signal to the final node that the previous node did not offer less value > > > for the HTLC than it was supposed to. (You could steal the top bit to > > > signal partial payment if you really want to). > > > > If `incomplete_payment` flag is set, then final nodes must claim HTLCs only > > if: > > > > sum(incoming_htlc_amt) >= amt_to_pay > > > > No, because now you've lost assurance that thisparticular HTLC hasn't > been skimmed by the previous node. > > ie. if I suspect a payment is using Base-AMP (and that's pretty clear if > I see two identical payment_hashes), I can reduce the amount I offer in > the outgoing HTLC to 1 satoshi: if it doesn't fail immediately, the next > hop is the final destination. > > > Where `sum(incoming_htlc_amt)` is the total `incoming_htlc_amt` for all > > incoming HTLCs terminating at this final node with the same `payment_hash`. > > But it's unnecessary for the recipient to know the total amount I meant > to pay; they just need to return the receipt once it exceeds the amount > they want. > > Cheers, > Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Seems like we can restrict the changes to BOLT11 by having the receiver assume NAMP for incoming payments < invoice_amount. (with some timeout of course, but that would need to be the case even when the sender is signalling NAMP). Cheers, Johan On Wed, Nov 21, 2018 at 3:55 AM ZmnSCPxj via Lightning-dev < lightning-dev@lists.linuxfoundation.org> wrote: > Good morning Rusty, > > > And do not play with `amount_to_forward`, as it's an important > > signal to the final node that the previous node did not offer less value > > for the HTLC than it was supposed to. (You could steal the top bit to > > signal partial payment if you really want to). > > I do not view this as playing with the existing `amt_to_forward`, but > rather retaining its previous use. > > If it helps, we can rewrite the *current* pre-AMP spec as below: > > 2. data: > ... > * [`8` : `amt_to_forward` / `amt_to_pay`] > > ... > > * `amt_to_forward` - for **non-final** nodes, this is the value to forward > to the next node. > Non-final nodes MUST check: > > incoming_htlc_amt - fee >= amt_to_forward > > * `amt_to_pay` - for **final** nodes, this is the value that is intended > to reach it. > Final nodes MUST check: > > incoming_htlc_amt >= amt_to_pay > > Then for Base AMP: > > * `amt_to_pay` - for **final** nodes, this is the total value that is > intended to reach it. > If `incomplete_payment` flag is not set, final nodes MUST check: > > incoming_htlc_amt >= amt_to_pay > > If `incomplete_payment` flag is set, then final nodes must claim HTLCs > only if: > > sum(incoming_htlc_amt) >= amt_to_pay > > Where `sum(incoming_htlc_amt)` is the total `incoming_htlc_amt` for all > incoming HTLCs terminating at this final node with the same `payment_hash`. > > > > Now perhaps we can argue that for AMP we should have two fields > `amt_to_pay_for_this_partial_payment` and `amt_to_pay_for_total_payment` > instead. > > > Regards, > ZmnSCPxj > > ___ > Lightning-dev mailing list > Lightning-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning Rusty, > And do not play with `amount_to_forward`, as it's an important > signal to the final node that the previous node did not offer less value > for the HTLC than it was supposed to. (You could steal the top bit to > signal partial payment if you really want to). I do not view this as playing with the existing `amt_to_forward`, but rather retaining its previous use. If it helps, we can rewrite the *current* pre-AMP spec as below: 2. data: ... * [`8` : `amt_to_forward` / `amt_to_pay`] ... * `amt_to_forward` - for **non-final** nodes, this is the value to forward to the next node. Non-final nodes MUST check: incoming_htlc_amt - fee >= amt_to_forward * `amt_to_pay` - for **final** nodes, this is the value that is intended to reach it. Final nodes MUST check: incoming_htlc_amt >= amt_to_pay Then for Base AMP: * `amt_to_pay` - for **final** nodes, this is the total value that is intended to reach it. If `incomplete_payment` flag is not set, final nodes MUST check: incoming_htlc_amt >= amt_to_pay If `incomplete_payment` flag is set, then final nodes must claim HTLCs only if: sum(incoming_htlc_amt) >= amt_to_pay Where `sum(incoming_htlc_amt)` is the total `incoming_htlc_amt` for all incoming HTLCs terminating at this final node with the same `payment_hash`. Now perhaps we can argue that for AMP we should have two fields `amt_to_pay_for_this_partial_payment` and `amt_to_pay_for_total_payment` instead. Regards, ZmnSCPxj ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
René Pickhardt via Lightning-dev writes: > Hey List, > > as this base AMP proposal seems pretty small I just started to write this > up to make a PR for BOLT04 and BOLT11. While doing my write up I realize > that there are smaller things that I would want to verify / double check > and propose with you. > > ## Verifying: > 1.) I understand the receiving node signals support for Base AMP by setting > a feature bit in the BOLT11 String Yes, this seemed a logical reason to add features to BOLT11. > 2.) The sending node signals a multipath payment by setting a feature bit > and by using the same `amount to forward` value in the last hop of the > onion for all paths which will also be bigger that the incoming htlcs whose > sum has to be at least the size of `amount_to_forward`. Not a feature bit as such, but some signal for the final node (in the onion). And do not play with `amount_to_forward`, as it's an important signal to the final node that the previous node did not offer less value for the HTLC than it was supposed to. (You could steal the top bit to signal partial payment if you really want to). > ## Clarifying: > 3.) Senders MUST NOT (SHOULD NOT?) create paths which would have to be > merged by intermediary nodes (as we don't know - and have no means of > querying - if they support the format of the adepted onion packages for > partial paths. Also it even seems impossible since the rest of the path for > at least one partial path could not be stored in the onion / forwarded > onions can't be seen) In-path merging is overreach, let's not do it or mention it. There's a slight preference to avoid sharing intermediary nodes to avoid correlation. Intermediary nodes know they need to forward all of them or not get paid for any of them, and they're already supposed to do so. > ## Proposing: > Should we specify an algorithm for executing a multipath payment for the > sending node or should this be left to the implementation. An obvious Idea > for an algorithm would be a divide and conquer scheme which should be > obvious with the following python style pseudo code: > > def pay_base_amp(amount): >success = False >for route in get_available_routes(): >success = send_via_route(route, amount) > if not success: >pay_base_amp(amount/2 + 1) # the +1 is to mitigate rounding errors. > there could be other ways to do so. >pay_base_amp(amount/2 + 1) I don't think this is actually useful. For example, I would suggest a more random split, and start by using some estimate of channel capacity. > Even if we leave the exact AMP execution to the sender we could still > suggest this divide and conquer scheme in BOLT 04 > > Another idea I had (which is probably a bad one as it allows for probing of > channel balances) would be to allow nodes on a partial path to send back > some hints of how much additional capacity they can forward if they see > that the partial payment feature bit is set (this would require to set this > feature bit in every onion) Also if we want to make use of this information > every node would have to support base amp. So I guess this idea is bad for > several reasons. Still we could have a MAY rule out of it? I think we should adapt a convention for a lower limit at which we disable the channel if we can't forward (eg 1% of capacity? 100 satoshis?). That gives us a better starting point for AMP, too. Cheers, Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning Rene, > Hey List, > > as this base AMP proposal seems pretty small I just started to write this up > to make a PR for BOLT04 and BOLT11. While doing my write up I realize that > there are smaller things that I would want to verify / double check and > propose with you. > > ## Verifying: > 1.) I understand the receiving node signals support for Base AMP by setting a > feature bit in the BOLT11 String Correct. > 2.) The sending node signals a multipath payment by setting a feature bit and > by using the same `amount to forward` value in the last hop of the onion for > all paths which will also be bigger that the incoming htlcs whose sum has to > be at least the size of `amount_to_forward`. The bit is not a "feature bit" specifically, as it is not a feature --- instead, it is a mode of operation or option. Perhaps semantics only, however. Otherwise, correct. > ## Clarifying: > 3.) Senders MUST NOT (SHOULD NOT?) create paths which would have to be merged > by intermediary nodes (as we don't know - and have no means of querying - if > they support the format of the adepted onion packages for partial paths. MUST NOT, since an incomplete payment can only be signaled for final nodes. I can explicitly use MUST NOT here I suppose. > Also it even seems impossible since the rest of the path for at least one > partial path could not be stored in the onion / forwarded onions can't be > seen) It would be easily doable to have the rest of the onion be the same for each incoming partial path. An intermediate hop just needs to store one onion, and compare the HMAC of a second or third incoming onion to differentiate between different forward-merges. > ## Proposing: > Should we specify an algorithm for executing a multipath payment for the > sending node or should this be left to the implementation. An obvious Idea > for an algorithm would be a divide and conquer scheme which should be obvious > with the following python style pseudo code: > > def pay_base_amp(amount): >success = False >for route in get_available_routes(): >success = send_via_route(route, amount) > if not success: >pay_base_amp(amount/2 + 1) # the +1 is to mitigate rounding errors. > there could be other ways to do so. >pay_base_amp(amount/2 + 1) > > Even if we leave the exact AMP execution to the sender we could still suggest > this divide and conquer scheme in BOLT 04 The above naive scheme will not work in the general case. See: https://lists.ozlabs.org/pipermail/c-lightning/2018-November/84.html It will not work well with the proposed `test_amp_unequal`, `test_amp_3way`. and `test_amp_5way` tests, at least until the amount has been split into tiny parts, possibly with fees becoming an issue (particularly `fee_base_msat`) due to having been split into many tiny parts. Regards, ZmnSCPxj___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Hey List, as this base AMP proposal seems pretty small I just started to write this up to make a PR for BOLT04 and BOLT11. While doing my write up I realize that there are smaller things that I would want to verify / double check and propose with you. ## Verifying: 1.) I understand the receiving node signals support for Base AMP by setting a feature bit in the BOLT11 String 2.) The sending node signals a multipath payment by setting a feature bit and by using the same `amount to forward` value in the last hop of the onion for all paths which will also be bigger that the incoming htlcs whose sum has to be at least the size of `amount_to_forward`. ## Clarifying: 3.) Senders MUST NOT (SHOULD NOT?) create paths which would have to be merged by intermediary nodes (as we don't know - and have no means of querying - if they support the format of the adepted onion packages for partial paths. Also it even seems impossible since the rest of the path for at least one partial path could not be stored in the onion / forwarded onions can't be seen) ## Proposing: Should we specify an algorithm for executing a multipath payment for the sending node or should this be left to the implementation. An obvious Idea for an algorithm would be a divide and conquer scheme which should be obvious with the following python style pseudo code: def pay_base_amp(amount): success = False for route in get_available_routes(): success = send_via_route(route, amount) if not success: pay_base_amp(amount/2 + 1) # the +1 is to mitigate rounding errors. there could be other ways to do so. pay_base_amp(amount/2 + 1) Even if we leave the exact AMP execution to the sender we could still suggest this divide and conquer scheme in BOLT 04 Another idea I had (which is probably a bad one as it allows for probing of channel balances) would be to allow nodes on a partial path to send back some hints of how much additional capacity they can forward if they see that the partial payment feature bit is set (this would require to set this feature bit in every onion) Also if we want to make use of this information every node would have to support base amp. So I guess this idea is bad for several reasons. Still we could have a MAY rule out of it? best Rene On Fri, Nov 16, 2018 at 4:45 PM Anthony Towns wrote: > On Thu, Nov 15, 2018 at 11:54:22PM +, ZmnSCPxj via Lightning-dev wrote: > > The improvement is in a reduction in `fee_base_msat` in the C->D path. > > I think reliability (and simplicity!) are the biggest things to improve > in lightning atm. Having the flag just be incuded in invoices and not > need to be gossiped seems simpler to me; and I think endpoint-only > merging is better for reliability too. Eg, if you find candidate routes: > > A -> B -> M -- actual directed capacity $6 > A -> C -> M -- actual directed capacity $5.50 > M -> E -> F -- actual directed capacity $6 > A -> X -> F -- actual directed capacity $7 > > and want to send $9 form A to F, you might start by trying to send > $5 via B and $4 via C. > > With endpoint-only merging you'd do: > >$5 via A,B,M,E,F -- partial success >$4 via A,C,M,E -- failure >$4 via A,X,F -- payment completion > > whereas with in-route merging, you'd do: > >$5 via A,B,M -- held >$4 via A,C,M -- to be continued >$9 via M,E -- both partial payments fail > > which seems a fair bit harder to incrementally recover from. > > > Granted, current `fee_base_msat` across the network is very low > currently. > > So I do not object to restricting merge points to ultimate payees. > > If fees rise later, we can revisit this. > > So, while we already agree on the approach to take, I think the above > provides an additional rationale :) > > Cheers, > aj > > ___ > Lightning-dev mailing list > Lightning-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > -- https://www.rene-pickhardt.de Skype: rene.pickhardt mobile: +49 (0)176 5762 3618 ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
On Thu, Nov 15, 2018 at 11:54:22PM +, ZmnSCPxj via Lightning-dev wrote: > The improvement is in a reduction in `fee_base_msat` in the C->D path. I think reliability (and simplicity!) are the biggest things to improve in lightning atm. Having the flag just be incuded in invoices and not need to be gossiped seems simpler to me; and I think endpoint-only merging is better for reliability too. Eg, if you find candidate routes: A -> B -> M -- actual directed capacity $6 A -> C -> M -- actual directed capacity $5.50 M -> E -> F -- actual directed capacity $6 A -> X -> F -- actual directed capacity $7 and want to send $9 form A to F, you might start by trying to send $5 via B and $4 via C. With endpoint-only merging you'd do: $5 via A,B,M,E,F -- partial success $4 via A,C,M,E -- failure $4 via A,X,F -- payment completion whereas with in-route merging, you'd do: $5 via A,B,M -- held $4 via A,C,M -- to be continued $9 via M,E -- both partial payments fail which seems a fair bit harder to incrementally recover from. > Granted, current `fee_base_msat` across the network is very low currently. > So I do not object to restricting merge points to ultimate payees. > If fees rise later, we can revisit this. So, while we already agree on the approach to take, I think the above provides an additional rationale :) Cheers, aj ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
I'm not sure this is an improvement at all over just allowing a single merge-point, i.e., the destination. You see as long as we don't attempt intermediate merges the routes are independent and failures of one HTLC do not impact any other parts. Take for example the network below: /\ AB-C-D \ / ---E--- For simplicity let's assume unit capacities on all channels except C-D and a total payment of 2 from A to D. If we use C as a merge point for the two partial payments A-C-D and A-B-C-D, then C can only forward if both partial payment succeed, i.e., if for example A-C fails then we'll need to tear down the HTLCs for both paths because it'll no longer be possible to find an alternative route to fulfill the forwarding of 2 over C-D. If however we have two independent routes A-B-C-D and A-C-D, then A-C-D can fail independently and we can recover by attempting A-E-D, no need to touch A-B-C-D at all. Overall it seems we get very little benefit (we save some HTLC setups and teardown) for a lot of added complexity. In the above case we would have saved on a single C-D HTLC, and the cost of doing so is many times larger (2 HTLCs needed to be torn down because we could no longer pass enough capacity to C in order for it to reach the forward threshold). Let's please stick with the simple mechanism of having the recipient be the only merge point. Cheers, Christian ZmnSCPxj via Lightning-dev writes: > Good morning list, > > I propose the below to support Base AMP. > > The below would allow arbitrary merges of paths, but not arbitrary splits. I > am uncertain about the safety of arbitrary splits. > > ### The `multipath_merge_per_hop` type (`option_base_amp`) > > This indicates that payment has been split by the sender using Base AMP, and > that the receiver should wait for the total intended payment before > forwarding or claiming the payment. > In case the receiving node is not the last node in the path, then succeeding > hops MUST be the same across all splits. > > 1. type: 1 (`termination_per_hop`) > 2. data: > * [`8` : `short_channel_id`] > * [`8` : `amt_to_forward`] > * [`4` : `outgoing_cltv_value`] > * [`8` : `intended_total_payment`] > * [`4` : `zeros`] > > The contents of this hop will be the same across all paths of the Base AMP. > The `payment_hash` of the incoming HTLCs will also be the same across all > paths of the Base AMP. > > `intended_total_payment` is the total amount of money that this node should > expect to receive in all incoming paths to the same `payment_hash`. > > This may be the last hop of a payment onion, in which case the `HMAC` for > this hop will be `0` (the same rule as for `per_hop_type` 0). > > The receiver: > > * MUST impose a reasonable timeout for waiting to receive all component > paths, and fail all incoming HTLC offers for the `payment_hash` if they have > not totalled equal to `intended_total_payment`. > * MUST NOT forward (if an intermediate node) or claim (if the final node) > unless it has received a total greater or equal to `intended_total_payment` > in all incoming HTLCs for the same `payment_hash`. > > The sender: > > * MUST use the same `payment_hash` for all paths of a single multipath > payment. > > Regards, > ZmnSCPxj > ___ > Lightning-dev mailing list > Lightning-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning Christian, The improvement is in a reduction in `fee_base_msat` in the C->D path. If C is the merge point, then the C->D `fee_base_msat` is only paid once, not twice. In effect, A is rationally choosing between a lower fee and better payment reliability. Granted, current `fee_base_msat` across the network is very low currently. So I do not object to restricting merge points to ultimate payees. If fees rise later, we can revisit this. Regards, ZmnSCPxj Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Friday, November 16, 2018 3:46 AM, Christian Decker wrote: > I'm not sure this is an improvement at all over just allowing a single > merge-point, i.e., the destination. You see as long as we don't attempt > intermediate merges the routes are independent and failures of one HTLC > do not impact any other parts. Take for example the network below: > > -- > > / \ > AB-C-D > \ / > ---E--- > > For simplicity let's assume unit capacities on all channels except C-D > and a total payment of 2 from A to D. > > If we use C as a merge point for the two partial payments A-C-D and > A-B-C-D, then C can only forward if both partial payment succeed, i.e., > if for example A-C fails then we'll need to tear down the HTLCs for both > paths because it'll no longer be possible to find an alternative route > to fulfill the forwarding of 2 over C-D. > > If however we have two independent routes A-B-C-D and A-C-D, then A-C-D > can fail independently and we can recover by attempting A-E-D, no need > to touch A-B-C-D at all. > > Overall it seems we get very little benefit (we save some HTLC setups > and teardown) for a lot of added complexity. In the above case we would > have saved on a single C-D HTLC, and the cost of doing so is many times > larger (2 HTLCs needed to be torn down because we could no longer pass > enough capacity to C in order for it to reach the forward threshold). > > Let's please stick with the simple mechanism of having the recipient be > the only merge point. > > Cheers, > Christian > > ZmnSCPxj via Lightning-dev lightning-dev@lists.linuxfoundation.org > writes: > > > Good morning list, > > I propose the below to support Base AMP. > > The below would allow arbitrary merges of paths, but not arbitrary splits. > > I am uncertain about the safety of arbitrary splits. > > > > ### The `multipath_merge_per_hop` type (`option_base_amp`) > > > > This indicates that payment has been split by the sender using Base AMP, > > and that the receiver should wait for the total intended payment before > > forwarding or claiming the payment. > > In case the receiving node is not the last node in the path, then > > succeeding hops MUST be the same across all splits. > > > > 1. type: 1 (`termination_per_hop`) > > 2. data: > > > > - [`8` : `short_channel_id`] > > - [`8` : `amt_to_forward`] > > - [`4` : `outgoing_cltv_value`] > > - [`8` : `intended_total_payment`] > > - [`4` : `zeros`] > > > > The contents of this hop will be the same across all paths of the Base AMP. > > The `payment_hash` of the incoming HTLCs will also be the same across all > > paths of the Base AMP. > > `intended_total_payment` is the total amount of money that this node should > > expect to receive in all incoming paths to the same `payment_hash`. > > This may be the last hop of a payment onion, in which case the `HMAC` for > > this hop will be `0` (the same rule as for `per_hop_type` 0). > > The receiver: > > > > - MUST impose a reasonable timeout for waiting to receive all component > > paths, and fail all incoming HTLC offers for the `payment_hash` if they > > have not totalled equal to `intended_total_payment`. > > - MUST NOT forward (if an intermediate node) or claim (if the final node) > > unless it has received a total greater or equal to `intended_total_payment` > > in all incoming HTLCs for the same `payment_hash`. > > > > The sender: > > > > - MUST use the same `payment_hash` for all paths of a single multipath > > payment. > > > > Regards, > > ZmnSCPxj > > > > Lightning-dev mailing list > > Lightning-dev@lists.linuxfoundation.org > > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning list, In case it was not noticed, I made a pull request for Base AMP: https://github.com/lightningnetwork/lightning-rfc/pull/511 This is primarily based on what Rusty suggested on-list, with sufficient MUST and SHOULD. Regards, ZmnSCPxj Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, November 14, 2018 9:59 AM, ZmnSCPxj via Lightning-dev wrote: > Good morning Rusty, > > Someone pointed out to me that `intended_payment_amount` is unnecessary. > On reflection, this is correct. > Both intermediate nodes and the payee node need not have > `intended_payment_amount`. > > Therefore > > > > I propose the below to support Base AMP. > > > > I think the complexity outweighs the benefits for the moment. The > > sender would have to make the onions identical past the merge point (so > > any one of them could be used), the merge point has to now create a > > many:1 map for HTLC redemption. > > For the moment, I think we should stick with: > > BOLT #4: > > > > 1. type: `per_hop` > > 2. data: > > - [`8`:`short_channel_id`] > > - [`8`:`amt_to_forward`] > > - [`4`:`outgoing_cltv_value`] > > > > - - [`12`:`padding`] > > - - [`1`:`flags`] > > - - [`11`:`padding`] > > And define bit 0 of `flags` as `incomplete_payment`. For the > > moment, it > > is only allowed for final nodes, and only if they put it in their > > BOLT11 > > field. > > > > We can do something even simpler. > > If `amt_to_forward` plus the fees charged by this node is greater than the > actual incoming HTLC, this is an AMP attempt. > No additional flag needs to be added. > For final payment nodes, if the `amt_to_forward` is greater than the incoming > HTLC value, this is an AMP attempt. > > The previous node could try to probe this by offering a smaller amount than > it was instructed to give, but cannot differentiate between a stall because > the payee is waiting for an AMP, or a stall due to some other unrelated error. > > -- > > Of course, an explicit flag is more sensible as it is more explicit. > > For myself, I would rather a new `per_hop_type`, but whether to use a > separate `per_hop_type` vs a byte in padding is largely a bikeshed issue and > either is fine with me. > A concern is that nothing in our current spec requires that `padding` be all > 0, such that reinterpreting byte 0 to be flags could cause interoperability > problems. > So perhaps a new `per_hop_type` which has a 2-byte `flags` (for more future > expansion) and a `padding` of 10 bytes which MUST be explicitly specced as > "MUST be all 0". > > An explicit flags field would also allow delivery of higher-layer application > data in each payment, for whatever purpose a higher-layer application may > want. E.g. bit 1 could mean "the next hop 65 bytes is actually a 32-byte > application ID and a 33-byte payload; this flag is valid only if this is the > last hop." > Another bit can also be used to provide spontaneous payment, so e.g. bit 2 > could mean "this hop is the final hop (even if HMAC is nonzero); the HMAC of > this hop is really the preimage to claim this payment." > > Regards, > ZmnSCPxj > > Lightning-dev mailing list > Lightning-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning Conner, > > MUST NOT forward (if an intermediate node) or claim (if the final node) > > unless > > it has received a total greater or equal to `intended_total_payment` in all > > incoming HTLCs for the same `payment_hash`. > > I was under the impression that this would not require changes on behalf of > the > intermediaries, and only need to be implemented by the sender and receiver? Strictly, it needs to be implemented by the sender and any merge points it wants. We have been thinking in terms of the receiver being the merge point, but it would be possible, with this scheme, for the merge point to be anywhere along the paths to the receiver. A-->B--->C--- \ \ ->D--->E--->F--- \ \ >G--->H--->I--->J In the above, the ultimate payee is J, which is a merge point. F is an intermediate node, but merges two paths together before forwarding. Other intermediate nodes, B, C, D, E, G, H, and I, are not merge points and do not need to understand this. Only F and J need to be given some merge point information in the new `per_hop_type`. B, C, D, E, G, H, and I, will be given `per_hop_type` of 0. This also means that AMP can be performed without the ultimate payee being AMP-capable, as the below: A-->B-->C--- \ \ ->D-->E--->F-->G Splitting the entire payment needs to be done at the ultimate source always, but merging can be done at any point along the way. So yes, it would be valuable to advertise the ability to merge payments as a global feature bit, not on the invoice. Regards, ZmnSCPxj > If not, then nodes would need to advertise that they support this so that the > sender can be sure to route through the subset of nodes that support it. > > Either way, it would seem that this constraint can only be accurately enforced > by the receiver. If any partial payments fail, then the > `intended_total_payment` > through an intermediary may never arise and the payment would be held. This > would also seem to exclude the possibility of iterative path finding, since > the > entire payment flow must be known up front during onion packet construction. > > Seems the proposal still works without the intermediaries needing to know > this? > > We may want to add that the receiver: > > - SHOULD fail the payment if `intended_total_payment` is less than the > invoice > amount > > > > I'm wondering, since these payments are no longer atomic, should we name it > > accordingly? > > Indeed this true. Perhaps NAMP or CPHR (Concurrent Payment Hash Re-use) are > more > accurate and may avoid confusion? > > Cheers, > Conner > On Tue, Nov 13, 2018 at 8:33 AM Johan Torås Halseth joha...@gmail.com wrote: > > > Good evening Z and list, > > I'm wondering, since these payments are no longer atomic, should we name it > > accordingly? > > Cheers, > > Johan > > On Tue, Nov 13, 2018 at 1:28 PM ZmnSCPxj via Lightning-dev > > lightning-dev@lists.linuxfoundation.org wrote: > > > > > Good morning list, > > > I propose the below to support Base AMP. > > > The below would allow arbitrary merges of paths, but not arbitrary > > > splits. I am uncertain about the safety of arbitrary splits. > > > > > > ### The `multipath_merge_per_hop` type (`option_base_amp`) > > > > > > This indicates that payment has been split by the sender using Base AMP, > > > and that the receiver should wait for the total intended payment before > > > forwarding or claiming the payment. > > > In case the receiving node is not the last node in the path, then > > > succeeding hops MUST be the same across all splits. > > > > > > 1. type: 1 (`termination_per_hop`) > > > 2. data: > > > > > > - [`8` : `short_channel_id`] > > > - [`8` : `amt_to_forward`] > > > - [`4` : `outgoing_cltv_value`] > > > - [`8` : `intended_total_payment`] > > > - [`4` : `zeros`] > > > > > > The contents of this hop will be the same across all paths of the Base > > > AMP. > > > The `payment_hash` of the incoming HTLCs will also be the same across all > > > paths of the Base AMP. > > > `intended_total_payment` is the total amount of money that this node > > > should expect to receive in all incoming paths to the same `payment_hash`. > > > This may be the last hop of a payment onion, in which case the `HMAC` for > > > this hop will be `0` (the same rule as for `per_hop_type` 0). > > > The receiver: > > > > > > - MUST impose a reasonable timeout for waiting to receive all component > > > paths, and fail all incoming HTLC offers for the `payment_hash` if they > > > have not totalled equal to `intended_total_payment`. > > > - MUST NOT forward (if an intermediate node) or claim (if the final > > > node) unless it has received a total greater or equal to > > > `intended_total_payment` in all incoming HTLCs for the same > > > `payment_hash`. > > > > > > The sender: > > > > > > - MUST use the
Re: [Lightning-dev] Base AMP
Good morning all, > MUST NOT forward (if an intermediate node) or claim (if the final node) unless > it has received a total greater or equal to `intended_total_payment` in all > incoming HTLCs for the same `payment_hash`. I was under the impression that this would not require changes on behalf of the intermediaries, and only need to be implemented by the sender and receiver? If not, then nodes would need to advertise that they support this so that the sender can be sure to route through the subset of nodes that support it. Either way, it would seem that this constraint can only be accurately enforced by the receiver. If any partial payments fail, then the `intended_total_payment` through an intermediary may never arise and the payment would be held. This would also seem to exclude the possibility of iterative path finding, since the entire payment flow must be known up front during onion packet construction. Seems the proposal still works without the intermediaries needing to know this? We may want to add that the receiver: * SHOULD fail the payment if `intended_total_payment` is less than the invoice amount > I'm wondering, since these payments are no longer atomic, should we name it > accordingly? Indeed this true. Perhaps NAMP or CPHR (Concurrent Payment Hash Re-use) are more accurate and may avoid confusion? Cheers, Conner On Tue, Nov 13, 2018 at 8:33 AM Johan Torås Halseth wrote: > > Good evening Z and list, > > I'm wondering, since these payments are no longer atomic, should we name it > accordingly? > > Cheers, > Johan > > On Tue, Nov 13, 2018 at 1:28 PM ZmnSCPxj via Lightning-dev > wrote: >> >> Good morning list, >> >> I propose the below to support Base AMP. >> >> The below would allow arbitrary merges of paths, but not arbitrary splits. >> I am uncertain about the safety of arbitrary splits. >> >> ### The `multipath_merge_per_hop` type (`option_base_amp`) >> >> This indicates that payment has been split by the sender using Base AMP, and >> that the receiver should wait for the total intended payment before >> forwarding or claiming the payment. >> In case the receiving node is not the last node in the path, then succeeding >> hops MUST be the same across all splits. >> >> 1. type: 1 (`termination_per_hop`) >> 2. data: >> * [`8` : `short_channel_id`] >> * [`8` : `amt_to_forward`] >> * [`4` : `outgoing_cltv_value`] >> * [`8` : `intended_total_payment`] >> * [`4` : `zeros`] >> >> The contents of this hop will be the same across all paths of the Base AMP. >> The `payment_hash` of the incoming HTLCs will also be the same across all >> paths of the Base AMP. >> >> `intended_total_payment` is the total amount of money that this node should >> expect to receive in all incoming paths to the same `payment_hash`. >> >> This may be the last hop of a payment onion, in which case the `HMAC` for >> this hop will be `0` (the same rule as for `per_hop_type` 0). >> >> The receiver: >> >> * MUST impose a reasonable timeout for waiting to receive all component >> paths, and fail all incoming HTLC offers for the `payment_hash` if they >> have not totalled equal to `intended_total_payment`. >> * MUST NOT forward (if an intermediate node) or claim (if the final node) >> unless it has received a total greater or equal to `intended_total_payment` >> in all incoming HTLCs for the same `payment_hash`. >> >> The sender: >> >> * MUST use the same `payment_hash` for all paths of a single multipath >> payment. >> >> Regards, >> ZmnSCPxj >> ___ >> Lightning-dev mailing list >> Lightning-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > > ___ > Lightning-dev mailing list > Lightning-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good evening Z and list, I'm wondering, since these payments are no longer atomic, should we name it accordingly? Cheers, Johan On Tue, Nov 13, 2018 at 1:28 PM ZmnSCPxj via Lightning-dev < lightning-dev@lists.linuxfoundation.org> wrote: > Good morning list, > > I propose the below to support Base AMP. > > The below would allow arbitrary merges of paths, but not arbitrary > splits. I am uncertain about the safety of arbitrary splits. > > ### The `multipath_merge_per_hop` type (`option_base_amp`) > > This indicates that payment has been split by the sender using Base AMP, > and that the receiver should wait for the total intended payment before > forwarding or claiming the payment. > In case the receiving node is not the last node in the path, then > succeeding hops MUST be the same across all splits. > > 1. type: 1 (`termination_per_hop`) > 2. data: > * [`8` : `short_channel_id`] > * [`8` : `amt_to_forward`] > * [`4` : `outgoing_cltv_value`] > * [`8` : `intended_total_payment`] > * [`4` : `zeros`] > > The contents of this hop will be the same across all paths of the Base AMP. > The `payment_hash` of the incoming HTLCs will also be the same across all > paths of the Base AMP. > > `intended_total_payment` is the total amount of money that this node > should expect to receive in all incoming paths to the same `payment_hash`. > > This may be the last hop of a payment onion, in which case the `HMAC` for > this hop will be `0` (the same rule as for `per_hop_type` 0). > > The receiver: > > * MUST impose a reasonable timeout for waiting to receive all component > paths, and fail all incoming HTLC offers for the `payment_hash` if they > have not totalled equal to `intended_total_payment`. > * MUST NOT forward (if an intermediate node) or claim (if the final node) > unless it has received a total greater or equal to `intended_total_payment` > in all incoming HTLCs for the same `payment_hash`. > > The sender: > > * MUST use the same `payment_hash` for all paths of a single multipath > payment. > > Regards, > ZmnSCPxj > ___ > Lightning-dev mailing list > Lightning-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
