Re: [Lightning-dev] A Proposal for Adding Bandwidth Metered Payment to Onion Messages
Hi Rusty, > AMP seems to be a Lightning Labs proprietary extension. You mean keysend, > which at least has a draft spec? I'm confused: the keyspend specification lives today as a very succinct bLIP [1] (use this record that contains the preimage settle the payment), and the AMP spec has had an open PR [2] (which admittedly needs to be mega-rebased) for sometime now and is amongst the oldest PRs in the BOLT tracker. Your definition of a "proprietary extension" here would seem to cover just about every open BOLT pull request that hasn't yet been widely adopted by multiple implementations. With that aside, AMP is used here instead of keysend as each payment split uses a different payment hash, which makes it harder to correlate payment splits as both the amount and hash will differ. Also given that the e2e payment needs to drop off additional fees (to pay for the forwarding pass) to each hop, the receiver doesn't actually know how much to expect as total payment (no invoice exists in this scenario). With keysend, it's possible the receiver accidentally pulls early, with the funds irrevocably sent, while no true onion messaging session has been constructed. With AMP the sender is the one that controls _when_ all the funds can be pulled, which eliminates such edge cases. > Sure, let's keep encouraging people to use HTLCs for free to send data? I > can certainly implement that if you prefer! I think the difference here is that HTLCs aren't that great for sending data: any space you allocate to arbitrary data takes away from the total route length. Also given the existence of the max_htlc parameters in channel updates, nodes can increase this value which allows them to discourage certain classes of behavior, as the HTLC that transmits the data serves effectively as an ephemeral bond, meaning that HTLC data transfer attempts incur an opportunity cost w.r.t the capital locked up. As a result, what's deployed in the wild today that uses HTLCs to send data (importantly at no additional marginal cost, since you sling 1.3KB across the wire anyway for normal payments), imo is mostly cases that only need a smaller amount of data, or use it mainly as a signalling layer. In contrast, onion messaging would allow users to send 32 KB (and beyond?), which makes certain classes of applications more feasible. > I suggest naively limiting to 10/sec for peers with channels, and 1/sec > for peers without for now. Curious to see what a fleshed out version of this would look like. IMO if this rate limiting event doesn't somehow make it back to the sender, then it isn't clear exactly why they're unable to even fetch an invoice to _attempt_ a payment. I think one issue with defaulting to always fetching invoices in a multi-hop manner is that a node failure may mean that a user can't even obtain an invoice, compared to just hitting an API or scanning a QR code that encodes one (pretty much 99% chance of success). (I think there's also a side discussion here w.r.t if tightly coupling an invoice/offer to a single node is compatible with the multi-node receive architectures that some services are beginning to develop/deploy, but we can make another thread for that.) > You mean: > 1. Don't use an even TLV field. I don't see why one _wouldn't_ use an even field here: if the node doesn't know what this means, then I want them to reject the payment, as otherwise they aren't able to properly do the bandwidth metering. Even assuming a feature bit, I think an even field would make sense, but ofc I'm open to further reasoning. > I think it's a premature optimization? Make standard duration 2016 > blocks; then they can request multiples if they want? Reduces > node_announcement size. Hmm, potentially but if I just need this to ask a node about their latest ephemeral onion service, do I really need the session to persist for 2016 blocks? At just 8 bytes here, it's smaller than a tor v3 onion address and even most feature bit distributions. Also there's nothing stopping someone from advertising optional feature bit 12,324 which means that nodes need to lug around and store all those extra bytes. > I'd be tempted to use 16 bytes? Collisions here are not really a thing > since you'd need a network packet per probe, and you're time limited. Yeah if the 32 bytes for each hop ends up eating too much into the total route length, then most use cases can get by with 16 bytes. > AFAICT this is easy to implement on top of onion_messages as they stand > today (if you don't want to fwd free onion messages at all, don't set > that bit?). Correct! The idea is to give nodes that don't want to forward for free an option to charge for forwarding and supplement their existing routing node revenue. On Twitter, Lisa brought up that as the data forwarding and payment aren't "atomic", you're reliant on the nodes to actually "do the thing" once the payment has been completed. This is correct, and imo can be mitigated mainly via a "tit for tat approac
Re: [Lightning-dev] A Proposal for Adding Bandwidth Metered Payment to Onion Messages
Olaoluwa Osuntokun writes: > Hi y'all, > > (TL;DR: a way to nodes to get paid to forward onion messages by adding an > upfront session creation phase that uses AMP tender a messaging session to a > receiver, with nodes being paid upfront for purchase of forwarding > bandwidth, and a session identifier being transmitted alongside onion > messages to identify paid sessions) AMP seems to be a Lightning Labs proprietary extension. You mean keysend, which at least has a draft spec? > Onion messaging has been proposed as a way to do things like fetch invoices > directly from a potential receiver _directly_ over the existing LN. The > current proposal (packaged under the BOLT 12 umbrella) uses a new message > (`onion_message`) that inherits the design of the existing Sphinx-based > onion blob included in htlc_add messages as a way to propagate arbitrary > messages across the network. Blinded paths which are effectively an unrolled > Sphinx SURB (single use reply block), are used to support reply messages in > a more private manner. Compared to SURBs, blinded paths are more flexible as > they don't lock in things like fees or CLTV values. > > A direct outcome of widespread adoption of the proposal is that the scope of > LN is expanded beyond "just" a decentralized p2p payment system, with the Sure, let's keep encouraging people to use HTLCs for free to send data? I can certainly implement that if you prefer! > 1. As there's no explicit session creation/acceptance, a node can be > spammed with unsolicited messages with no way to deny unwanted messages nor > explicitly allow messages from certain senders. > > 2. Nodes that forward these messages (up to 32 KB per message) receive no > compensation for the network bandwidth their expend, effectively shuffling > around messages for free. > > 3. Rate limiting isn't concretely addressed, which may result in > heterogeneous rate limiting policies enforced around the network, which can > degrade the developer/user experience (why are my packets being randomly > dropped?). Sure, this is a fun one! I can post separately on ratelimiting; I suggest naively limiting to 10/sec for peers with channels, and 1/sec for peers without for now. (In practice, spamming with HTLCs is infinitely more satisfying...) > In this email I propose a way to address the issues mentioned above by > adding explicit onion messaging session creation as well as a way for nodes > to be (optionally) paid for any onion messages they forward. In short, an > explicit session creation phase is introduced, with the receiver being able > to accept/deny the session. If the session is accepted, then all nodes that > comprise the session route are compensated for allotting a certain amount of > bandwidth to the session (which is ephemeral by nature). It's an interesting layer on top (esp if you want to stream movies), but I never proposed this because it seems to add a source-identifying session id, which is a huge privacy step backwards. You really *do not want* to use this for independent transmissions. I flirted with using blinded tokens, but it gets complex fast; ideas welcome! > ## Node Announcement TLV Extension > > In order to allow nodes to signal that they want to be paid to forward onion > messages and also specify their pricing, we add two new TLV to the node_ann > message: > > * type: 1 (`sats_per_byte`) >* data: > * [`uint64`:`forwarding_rate`] > * type: 2 (`sats_per_block`) >* data: > * [`uint64`:`per_block_rate`] You mean: * type: 1 (`sats_per_byte`) * data: * [`tu64`:`forwarding_rate`] * type: 3 (`sats_per_block`) * data: * [`tu64`:`per_block_rate`] 1. Don't use an even TLV field. 2. Might as well use truncated u64. > The `sats_per_byte` field allows nodes to price their bandwidth, ensuring > that they get paid for each chunk of allocated bandwidth. As sessions have a > fixed time frame and nodes need to store additional data within that time > frame, the `sats_per_block` allows nodes to price this cost, as they'll hold > onto the session identifier information until the specified block height > (detailed below). > > As onion messages will _typically_ be fixed sized we may want to use > coursers > metering here instead of bytes, possibly paying for 1.3KB or 32 KB chunks > instead. I think it's a premature optimization? Make standard duration 2016 blocks; then they can request multiples if they want? Reduces node_announcement size. > With the above nodes are able to express that they're willing to forward > messages for sats, and how much they charge per byte as well as per block. > Next we add a new TLV in the _existing_ HTLC onion blob that allows a > sending node to tender paid onion message session creation. A sketch of this > type would look something like: > > * type: 14 (`onion_session_id`) > * data: > * [`32*byte`:`session_id`] I'd be tempted to use 16 bytes? Collisions here are not really a thing
[Lightning-dev] A Proposal for Adding Bandwidth Metered Payment to Onion Messages
Hi y'all, (TL;DR: a way to nodes to get paid to forward onion messages by adding an upfront session creation phase that uses AMP tender a messaging session to a receiver, with nodes being paid upfront for purchase of forwarding bandwidth, and a session identifier being transmitted alongside onion messages to identify paid sessions) Onion messaging has been proposed as a way to do things like fetch invoices directly from a potential receiver _directly_ over the existing LN. The current proposal (packaged under the BOLT 12 umbrella) uses a new message (`onion_message`) that inherits the design of the existing Sphinx-based onion blob included in htlc_add messages as a way to propagate arbitrary messages across the network. Blinded paths which are effectively an unrolled Sphinx SURB (single use reply block), are used to support reply messages in a more private manner. Compared to SURBs, blinded paths are more flexible as they don't lock in things like fees or CLTV values. A direct outcome of widespread adoption of the proposal is that the scope of LN is expanded beyond "just" a decentralized p2p payment system, with the protocol evolving to also support pseudonymous messaging and arbitrary data transfer across the network. This expanded network (payments + arbitrary data transfer) enables use cases like streaming video transfer, network tunneled VPNs, large file download, popcorn time, etc -- . Depending on one's view, the existence of such a combined protocol/network may either elicit feelings of dread (can we really do _both_ payments _and_ data properly in the same network?) or excitement (I finally have a censorship resistant way to watch unboxing videos of all my favorite gadgets!). Putting aside the discussion w.r.t if such an expanded network is desirable and also if the combined functionality fundamentally _needs_ to exist in the confines of a single protocol stack (eg: if LN impls packaged tor clients would that be enough?), IMO onion messaging as currently proposed has a few issues: 1. As there's no explicit session creation/acceptance, a node can be spammed with unsolicited messages with no way to deny unwanted messages nor explicitly allow messages from certain senders. 2. Nodes that forward these messages (up to 32 KB per message) receive no compensation for the network bandwidth their expend, effectively shuffling around messages for free. 3. Rate limiting isn't concretely addressed, which may result in heterogeneous rate limiting policies enforced around the network, which can degrade the developer/user experience (why are my packets being randomly dropped?). In this email I propose a way to address the issues mentioned above by adding explicit onion messaging session creation as well as a way for nodes to be (optionally) paid for any onion messages they forward. In short, an explicit session creation phase is introduced, with the receiver being able to accept/deny the session. If the session is accepted, then all nodes that comprise the session route are compensated for allotting a certain amount of bandwidth to the session (which is ephemeral by nature). # High-Level Overview Inspired by HORNETs two-phase session creation (first phase makes the circuit, send allows data transfers), I propose we break up onion messaging session creation into two phases. In the first phase a sender purchases _forwarding bandwidth_ from a series of intermediate nodes and also requests creation of a messaging session to the receiver in a single _atomic_ step. In the second phase, assuming the session creation was successful, the sender is able to use the purchased forwarding bandwidth to send messages to the receiver. The session stays open until either it expires, or the receiver runs out of cumulative forwarding bandwidth and needs to repeat the first step. As we'll see shortly, the created onion messaging sessions aren't tightly coupled to the nodes that are a part of the initial session creation. Instead session creation creates a sort of overlay network from the PoV of the sender that can be used to transmit messages. The same route doesn't need to be used by subsequent onion message transmissions as the sending node may already have existing bandwidth sessions it can put together to send a new/existing message. One trade-off of the current approach is that a small amount of per-session state is added to nodes that want to be paid to forward onion messages. The current onion messaging proposal takes care to _not_ introduce any extra state to nodes in an onion messaging path: they just decrypt/unblinded and forward to the next hop. This proposal as it stands adds just 40-bytes-ish of storage overhead per session (which are ephemeral so this state can be forgotten over time). In practice, as nodes are being paid to forward, they can ensure their pricing (more on that later) properly compensates then for this added storage per session. # AMP + Onion Messaging == Paid Onion Messaging What
