Re: [Lightning-dev] [PATCH] First draft of option_simplfied_commitment
Not sure if others already realized this, but in thinking about our RBF policy hack from Adelaide a bit more, to allow the carve-out exception of "last tx in a package, which has only one unconfirmed ancestor" to always be available for the "honest party" when broadcasting a commitment transaction, we also need at least a CSV delay of 1 block on the HTLC transaction outputs (as otherwise those transactions could count as the carve-out tx). Matt On 11/21/18 2:17 AM, Rusty Russell wrote: I'm also starting to implement this, to see what I missed! Original at https://github.com/lightningnetwork/lightning-rfc/pull/513 Pasted here for your reading convenience: - Option is sticky; it set at open time, it stays with channel - I didn't want to have to handle penalty txs on channels which switch - We could, however, upgrade on splice. - Feerate is fixed at 253 - `feerate_per_kw` is still in open /accept (just ignored): multifund may want it. - closing tx negotiates *upwards* not *downwards* - Starting from base fee of commitment tx = 282 satoshi. - to_remote output is always CSV delayed. - pushme outputs are paid for by funder, but only exist if the matching to_local/remote output exists. - After 10 blocks, they become anyone-can-spend (they need to see the to-local/remote witness script though). - remotepubkey is not rotated. - You must spend your pushme output; you may sweep for others. Signed-off-by: Rusty Russell diff --git a/02-peer-protocol.md b/02-peer-protocol.md index 7cf9ebf..6ec1155 100644 --- a/02-peer-protocol.md +++ b/02-peer-protocol.md @@ -133,7 +133,9 @@ node can offer. (i.e. 1/4 the more normally-used 'satoshi per 1000 vbytes') that this side will pay for commitment and HTLC transactions, as described in [BOLT #3](03-transactions.md#fee-calculation) (this can be adjusted -later with an `update_fee` message). +later with an `update_fee` message). Note that if +`option_simplified_commitment` is negotiated, this `feerate_per_kw` +is treated as 253 for all transactions. `to_self_delay` is the number of blocks that the other node's to-self outputs must be delayed, using `OP_CHECKSEQUENCEVERIFY` delays; this @@ -208,7 +210,8 @@ The receiving node MUST fail the channel if: - `push_msat` is greater than `funding_satoshis` * 1000. - `to_self_delay` is unreasonably large. - `max_accepted_htlcs` is greater than 483. - - it considers `feerate_per_kw` too small for timely processing or unreasonably large. + - if `option_simplified_commitment` is not negotiated: +- it considers `feerate_per_kw` too small for timely processing or unreasonably large. - `funding_pubkey`, `revocation_basepoint`, `htlc_basepoint`, `payment_basepoint`, or `delayed_payment_basepoint` are not valid DER-encoded compressed secp256k1 pubkeys. - `dust_limit_satoshis` is greater than `channel_reserve_satoshis`. @@ -228,7 +231,7 @@ The *channel reserve* is specified by the peer's `channel_reserve_satoshis`: 1% The sender can unconditionally give initial funds to the receiver using a non-zero `push_msat`, but even in this case we ensure that the funder has sufficient remaining funds to pay fees and that one side has some amount it can spend (which also implies there is at least one non-dust output). Note that, like any other on-chain transaction, this payment is not certain until the funding transaction has been confirmed sufficiently (with a danger of double-spend until this occurs) and may require a separate method to prove payment via on-chain confirmation. -The `feerate_per_kw` is generally only of concern to the sender (who pays the fees), but there is also the fee rate paid by HTLC transactions; thus, unreasonably large fee rates can also penalize the recipient. +The `feerate_per_kw` is generally only of concern to the sender (who pays the fees), but there is also the fee rate paid by HTLC transactions; thus, unreasonably large fee rates can also penalize the recipient. It is ignored for `option_simplified_commitment`. Separating the `htlc_basepoint` from the `payment_basepoint` improves security: a node needs the secret associated with the `htlc_basepoint` to produce HTLC signatures for the protocol, but the secret for the `payment_basepoint` can be in cold storage. @@ -340,6 +343,12 @@ This message introduces the `channel_id` to identify the channel. It's derived f Requirements +Both peers: + - if `option_simplified_commitment` was negotiated: +- `option_simplified_commitment` applies to all commitment and HTLC transactions + - otherwise: +- `option_simplified_commitment` does not apply to any commitment or HTLC transactions + The sender MUST set: - `channel_id` by exclusive-OR of the `funding_txid` and the `funding_output_index` from the `funding_created` message. - `signature` to the valid signature, using its `funding_pubkey` for the initial commitment transaction, as defined in [BOLT
Re: [Lightning-dev] Base AMP
Good morning Rusty, > And do not play with `amount_to_forward`, as it's an important > signal to the final node that the previous node did not offer less value > for the HTLC than it was supposed to. (You could steal the top bit to > signal partial payment if you really want to). I do not view this as playing with the existing `amt_to_forward`, but rather retaining its previous use. If it helps, we can rewrite the *current* pre-AMP spec as below: 2. data: ... * [`8` : `amt_to_forward` / `amt_to_pay`] ... * `amt_to_forward` - for **non-final** nodes, this is the value to forward to the next node. Non-final nodes MUST check: incoming_htlc_amt - fee >= amt_to_forward * `amt_to_pay` - for **final** nodes, this is the value that is intended to reach it. Final nodes MUST check: incoming_htlc_amt >= amt_to_pay Then for Base AMP: * `amt_to_pay` - for **final** nodes, this is the total value that is intended to reach it. If `incomplete_payment` flag is not set, final nodes MUST check: incoming_htlc_amt >= amt_to_pay If `incomplete_payment` flag is set, then final nodes must claim HTLCs only if: sum(incoming_htlc_amt) >= amt_to_pay Where `sum(incoming_htlc_amt)` is the total `incoming_htlc_amt` for all incoming HTLCs terminating at this final node with the same `payment_hash`. Now perhaps we can argue that for AMP we should have two fields `amt_to_pay_for_this_partial_payment` and `amt_to_pay_for_total_payment` instead. Regards, ZmnSCPxj ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
René Pickhardt via Lightning-dev writes: > Hey List, > > as this base AMP proposal seems pretty small I just started to write this > up to make a PR for BOLT04 and BOLT11. While doing my write up I realize > that there are smaller things that I would want to verify / double check > and propose with you. > > ## Verifying: > 1.) I understand the receiving node signals support for Base AMP by setting > a feature bit in the BOLT11 String Yes, this seemed a logical reason to add features to BOLT11. > 2.) The sending node signals a multipath payment by setting a feature bit > and by using the same `amount to forward` value in the last hop of the > onion for all paths which will also be bigger that the incoming htlcs whose > sum has to be at least the size of `amount_to_forward`. Not a feature bit as such, but some signal for the final node (in the onion). And do not play with `amount_to_forward`, as it's an important signal to the final node that the previous node did not offer less value for the HTLC than it was supposed to. (You could steal the top bit to signal partial payment if you really want to). > ## Clarifying: > 3.) Senders MUST NOT (SHOULD NOT?) create paths which would have to be > merged by intermediary nodes (as we don't know - and have no means of > querying - if they support the format of the adepted onion packages for > partial paths. Also it even seems impossible since the rest of the path for > at least one partial path could not be stored in the onion / forwarded > onions can't be seen) In-path merging is overreach, let's not do it or mention it. There's a slight preference to avoid sharing intermediary nodes to avoid correlation. Intermediary nodes know they need to forward all of them or not get paid for any of them, and they're already supposed to do so. > ## Proposing: > Should we specify an algorithm for executing a multipath payment for the > sending node or should this be left to the implementation. An obvious Idea > for an algorithm would be a divide and conquer scheme which should be > obvious with the following python style pseudo code: > > def pay_base_amp(amount): >success = False >for route in get_available_routes(): >success = send_via_route(route, amount) > if not success: >pay_base_amp(amount/2 + 1) # the +1 is to mitigate rounding errors. > there could be other ways to do so. >pay_base_amp(amount/2 + 1) I don't think this is actually useful. For example, I would suggest a more random split, and start by using some estimate of channel capacity. > Even if we leave the exact AMP execution to the sender we could still > suggest this divide and conquer scheme in BOLT 04 > > Another idea I had (which is probably a bad one as it allows for probing of > channel balances) would be to allow nodes on a partial path to send back > some hints of how much additional capacity they can forward if they see > that the partial payment feature bit is set (this would require to set this > feature bit in every onion) Also if we want to make use of this information > every node would have to support base amp. So I guess this idea is bad for > several reasons. Still we could have a MAY rule out of it? I think we should adapt a convention for a lower limit at which we disable the channel if we can't forward (eg 1% of capacity? 100 satoshis?). That gives us a better starting point for AMP, too. Cheers, Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Invoice Address Format
Varunram Ganesh writes: > Now, I am no expert on error encoding formats, but I think that bech32 is > under optimised for invoices (whose lengths are greater than 71). Related to > this, is there a reason why we use hex encoded pubkeys in lightning? Unless > I'm missing something, I think bech32 is better to use in this context. > Please correct me if I'm wrong. No, you're right. It was chosen because it's an existing encoding which already exists in the bitcoin ecosystem. The signature on an invoice provides a far stronger guarantee than the 6-char checksum anyway. Cheers, Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
[Lightning-dev] [PATCH] First draft of option_simplfied_commitment
I'm also starting to implement this, to see what I missed! Original at https://github.com/lightningnetwork/lightning-rfc/pull/513 Pasted here for your reading convenience: - Option is sticky; it set at open time, it stays with channel - I didn't want to have to handle penalty txs on channels which switch - We could, however, upgrade on splice. - Feerate is fixed at 253 - `feerate_per_kw` is still in open /accept (just ignored): multifund may want it. - closing tx negotiates *upwards* not *downwards* - Starting from base fee of commitment tx = 282 satoshi. - to_remote output is always CSV delayed. - pushme outputs are paid for by funder, but only exist if the matching to_local/remote output exists. - After 10 blocks, they become anyone-can-spend (they need to see the to-local/remote witness script though). - remotepubkey is not rotated. - You must spend your pushme output; you may sweep for others. Signed-off-by: Rusty Russell diff --git a/02-peer-protocol.md b/02-peer-protocol.md index 7cf9ebf..6ec1155 100644 --- a/02-peer-protocol.md +++ b/02-peer-protocol.md @@ -133,7 +133,9 @@ node can offer. (i.e. 1/4 the more normally-used 'satoshi per 1000 vbytes') that this side will pay for commitment and HTLC transactions, as described in [BOLT #3](03-transactions.md#fee-calculation) (this can be adjusted -later with an `update_fee` message). +later with an `update_fee` message). Note that if +`option_simplified_commitment` is negotiated, this `feerate_per_kw` +is treated as 253 for all transactions. `to_self_delay` is the number of blocks that the other node's to-self outputs must be delayed, using `OP_CHECKSEQUENCEVERIFY` delays; this @@ -208,7 +210,8 @@ The receiving node MUST fail the channel if: - `push_msat` is greater than `funding_satoshis` * 1000. - `to_self_delay` is unreasonably large. - `max_accepted_htlcs` is greater than 483. - - it considers `feerate_per_kw` too small for timely processing or unreasonably large. + - if `option_simplified_commitment` is not negotiated: +- it considers `feerate_per_kw` too small for timely processing or unreasonably large. - `funding_pubkey`, `revocation_basepoint`, `htlc_basepoint`, `payment_basepoint`, or `delayed_payment_basepoint` are not valid DER-encoded compressed secp256k1 pubkeys. - `dust_limit_satoshis` is greater than `channel_reserve_satoshis`. @@ -228,7 +231,7 @@ The *channel reserve* is specified by the peer's `channel_reserve_satoshis`: 1% The sender can unconditionally give initial funds to the receiver using a non-zero `push_msat`, but even in this case we ensure that the funder has sufficient remaining funds to pay fees and that one side has some amount it can spend (which also implies there is at least one non-dust output). Note that, like any other on-chain transaction, this payment is not certain until the funding transaction has been confirmed sufficiently (with a danger of double-spend until this occurs) and may require a separate method to prove payment via on-chain confirmation. -The `feerate_per_kw` is generally only of concern to the sender (who pays the fees), but there is also the fee rate paid by HTLC transactions; thus, unreasonably large fee rates can also penalize the recipient. +The `feerate_per_kw` is generally only of concern to the sender (who pays the fees), but there is also the fee rate paid by HTLC transactions; thus, unreasonably large fee rates can also penalize the recipient. It is ignored for `option_simplified_commitment`. Separating the `htlc_basepoint` from the `payment_basepoint` improves security: a node needs the secret associated with the `htlc_basepoint` to produce HTLC signatures for the protocol, but the secret for the `payment_basepoint` can be in cold storage. @@ -340,6 +343,12 @@ This message introduces the `channel_id` to identify the channel. It's derived f Requirements +Both peers: + - if `option_simplified_commitment` was negotiated: +- `option_simplified_commitment` applies to all commitment and HTLC transactions + - otherwise: +- `option_simplified_commitment` does not apply to any commitment or HTLC transactions + The sender MUST set: - `channel_id` by exclusive-OR of the `funding_txid` and the `funding_output_index` from the `funding_created` message. - `signature` to the valid signature, using its `funding_pubkey` for the initial commitment transaction, as defined in [BOLT #3](03-transactions.md#commitment-transaction). @@ -351,6 +360,12 @@ The recipient: - on receipt of a valid `funding_signed`: - SHOULD broadcast the funding transaction. + Rationale + +We decide on `option_simplified_commitment` at this point when we first have to generate the commitment +transaction. Even if a later reconnection does not negotiate this parameter, this channel will honor it. +This simplifies channel state, particularly penalty transaction handling. + ### The `funding_locked` Message
Re: [Lightning-dev] Base AMP
Hey List, as this base AMP proposal seems pretty small I just started to write this up to make a PR for BOLT04 and BOLT11. While doing my write up I realize that there are smaller things that I would want to verify / double check and propose with you. ## Verifying: 1.) I understand the receiving node signals support for Base AMP by setting a feature bit in the BOLT11 String 2.) The sending node signals a multipath payment by setting a feature bit and by using the same `amount to forward` value in the last hop of the onion for all paths which will also be bigger that the incoming htlcs whose sum has to be at least the size of `amount_to_forward`. ## Clarifying: 3.) Senders MUST NOT (SHOULD NOT?) create paths which would have to be merged by intermediary nodes (as we don't know - and have no means of querying - if they support the format of the adepted onion packages for partial paths. Also it even seems impossible since the rest of the path for at least one partial path could not be stored in the onion / forwarded onions can't be seen) ## Proposing: Should we specify an algorithm for executing a multipath payment for the sending node or should this be left to the implementation. An obvious Idea for an algorithm would be a divide and conquer scheme which should be obvious with the following python style pseudo code: def pay_base_amp(amount): success = False for route in get_available_routes(): success = send_via_route(route, amount) if not success: pay_base_amp(amount/2 + 1) # the +1 is to mitigate rounding errors. there could be other ways to do so. pay_base_amp(amount/2 + 1) Even if we leave the exact AMP execution to the sender we could still suggest this divide and conquer scheme in BOLT 04 Another idea I had (which is probably a bad one as it allows for probing of channel balances) would be to allow nodes on a partial path to send back some hints of how much additional capacity they can forward if they see that the partial payment feature bit is set (this would require to set this feature bit in every onion) Also if we want to make use of this information every node would have to support base amp. So I guess this idea is bad for several reasons. Still we could have a MAY rule out of it? best Rene On Fri, Nov 16, 2018 at 4:45 PM Anthony Towns wrote: > On Thu, Nov 15, 2018 at 11:54:22PM +, ZmnSCPxj via Lightning-dev wrote: > > The improvement is in a reduction in `fee_base_msat` in the C->D path. > > I think reliability (and simplicity!) are the biggest things to improve > in lightning atm. Having the flag just be incuded in invoices and not > need to be gossiped seems simpler to me; and I think endpoint-only > merging is better for reliability too. Eg, if you find candidate routes: > > A -> B -> M -- actual directed capacity $6 > A -> C -> M -- actual directed capacity $5.50 > M -> E -> F -- actual directed capacity $6 > A -> X -> F -- actual directed capacity $7 > > and want to send $9 form A to F, you might start by trying to send > $5 via B and $4 via C. > > With endpoint-only merging you'd do: > >$5 via A,B,M,E,F -- partial success >$4 via A,C,M,E -- failure >$4 via A,X,F -- payment completion > > whereas with in-route merging, you'd do: > >$5 via A,B,M -- held >$4 via A,C,M -- to be continued >$9 via M,E -- both partial payments fail > > which seems a fair bit harder to incrementally recover from. > > > Granted, current `fee_base_msat` across the network is very low > currently. > > So I do not object to restricting merge points to ultimate payees. > > If fees rise later, we can revisit this. > > So, while we already agree on the approach to take, I think the above > provides an additional rationale :) > > Cheers, > aj > > ___ > Lightning-dev mailing list > Lightning-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > -- https://www.rene-pickhardt.de Skype: rene.pickhardt mobile: +49 (0)176 5762 3618 ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Base AMP
Good morning Rene, > Hey List, > > as this base AMP proposal seems pretty small I just started to write this up > to make a PR for BOLT04 and BOLT11. While doing my write up I realize that > there are smaller things that I would want to verify / double check and > propose with you. > > ## Verifying: > 1.) I understand the receiving node signals support for Base AMP by setting a > feature bit in the BOLT11 String Correct. > 2.) The sending node signals a multipath payment by setting a feature bit and > by using the same `amount to forward` value in the last hop of the onion for > all paths which will also be bigger that the incoming htlcs whose sum has to > be at least the size of `amount_to_forward`. The bit is not a "feature bit" specifically, as it is not a feature --- instead, it is a mode of operation or option. Perhaps semantics only, however. Otherwise, correct. > ## Clarifying: > 3.) Senders MUST NOT (SHOULD NOT?) create paths which would have to be merged > by intermediary nodes (as we don't know - and have no means of querying - if > they support the format of the adepted onion packages for partial paths. MUST NOT, since an incomplete payment can only be signaled for final nodes. I can explicitly use MUST NOT here I suppose. > Also it even seems impossible since the rest of the path for at least one > partial path could not be stored in the onion / forwarded onions can't be > seen) It would be easily doable to have the rest of the onion be the same for each incoming partial path. An intermediate hop just needs to store one onion, and compare the HMAC of a second or third incoming onion to differentiate between different forward-merges. > ## Proposing: > Should we specify an algorithm for executing a multipath payment for the > sending node or should this be left to the implementation. An obvious Idea > for an algorithm would be a divide and conquer scheme which should be obvious > with the following python style pseudo code: > > def pay_base_amp(amount): >success = False >for route in get_available_routes(): >success = send_via_route(route, amount) > if not success: >pay_base_amp(amount/2 + 1) # the +1 is to mitigate rounding errors. > there could be other ways to do so. >pay_base_amp(amount/2 + 1) > > Even if we leave the exact AMP execution to the sender we could still suggest > this divide and conquer scheme in BOLT 04 The above naive scheme will not work in the general case. See: https://lists.ozlabs.org/pipermail/c-lightning/2018-November/84.html It will not work well with the proposed `test_amp_unequal`, `test_amp_3way`. and `test_amp_5way` tests, at least until the amount has been split into tiny parts, possibly with fees becoming an issue (particularly `fee_base_msat`) due to having been split into many tiny parts. Regards, ZmnSCPxj___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
[Lightning-dev] Invoice Address Format
Hello List, The reason why we use bech32 for invoice addresses and raw hex encoded pubkeys and has long puzzled me. Let's take a sample testnet invoice >>> "lntb20m1pvjluezhp58yjmdan79s6qqdhdzgynm4zwqd5d7xmw5fk98klysy043l2ahrqspp5qqqsyqcyq5rqwzqfqqqsyqcyq5rqwzqfqqqsyqcyq5rqwzqfqypqfpp3x9et2e20v6pu37c5d9vax37wxq72un98kmzzhznpurw9sgl2v0nklu2g4d0keph5t7tj9tcqd8rexnd07ux4uv2cjvcqwaxgj7v4uwn5wmypjd5n69z2xm3xgksg28nwht7f6zspwp3f9t" <<< of length 271 from the BOLT 11 RFC for consideration. While the bech32 format has its advantages being base 32, easy to read out over phone and stuff like that, I wonder if it is optimised for this specific use case since BIP-173 states >>> The specific code chosen here is the result of: 1. Starting with an exhaustive list of 159605 BCH codes designed to detect 3 or 4 errors up to length 93, 151, 165, 341, 1023, and 1057. 2. From those, requiring the detection of 4 errors up to length 71, resulting in 28825 remaining codes. <<< (what's important here is the second part on lengths) Now, I am no expert on error encoding formats, but I think that bech32 is under optimised for invoices (whose lengths are greater than 71). Related to this, is there a reason why we use hex encoded pubkeys in lightning? Unless I'm missing something, I think bech32 is better to use in this context. Please correct me if I'm wrong. Have a nice day, Varunram___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
