Re: [Lightning-dev] bLIPs: A proposal for community-driven app layer and protocol extension standardization
> bLIPs have a slightly different process than BIPs, as well as a different set > of editors/maintainers (more widely distributed). As we saw with the Speedy > Trial saga (fingers crossed), the sole (?) maintainer of the BIP process was > able to effectively steelman the progression of an author document, with no > sound technical objection (they had a competing proposal that could've been > a distinct document). bLIPs sidestep shenanigans like this by having the > primary maintainer/editors be more widely distributed and closer to the > target domain (LN). That's fair. The BIP repository as a whole lost credibility when the sole maintainer at the time steelmanned progress for no good reason. It was done in bad faith and no steps were taken to remove even the *perception* of impropriety. I hope the maintainers of the bLIP repository can be more impartial. The automatic assignment of proposal numbers is an excellent improvement. It's an idea that the BIP repo should adopt since it removes responsibility and power from the maintainers. > > The other thing bLIPs do is do away with the whole "human picks the number > of documents", and "don't assign your own number, you must wait". Borrowing > from EIPs, the number of a document is simply the number of the PR that > proposes the document. This reduces friction, and eliminates a possible > bikeshedding vector. > > -- Laolu > > > On Wed, Jun 30, 2021 at 5:31 PM Ariel Luaces wrote: >> >> BIPs are already the Bazaar style of evolution that simultaneously >> allows flexibility and coordination/interoperability (since anyone can >> create a BIP and they create an environment of discussion). >> >> BOLTs are essentially one big BIP in the sense that they started as a >> place for discussion but are now more rigid. BOLTs must be followed >> strictly to ensure a node is interoperable with the network. And BOLTs >> should be rigid, as rigid as any widely used BIP like 32 for example. >> Even though BOLTs were flexible when being drafted their purpose has >> changed from descriptive to prescriptive. >> Any alternatives, or optional features should be extracted out of >> BOLTs, written as BIPs. The BIP should then reference the BOLT and the >> required flags set, messages sent, or alterations made to signal that >> the BIP's feature is enabled. >> >> A BOLT may at some point organically change to reference a BIP. For >> example if a BIP was drafted as an optional feature but then becomes >> more widespread and then turns out to be crucial for the proper >> operation of the network then a BOLT can be changed to just reference >> the BIP as mandatory. There isn't anything wrong with this. >> >> All of the above would work exactly the same if there was a bLIP >> repository instead. I don't see the value in having both bLIPs and >> BIPs since AFAICT they seem to be functionally equivalent and BIPs are >> not restricted to exclude lightning, and never have been. >> >> I believe the reason this move to BIPs hasn't happened organically is >> because many still perceive the BOLTs available for editing, so >> changes continue to be made. If instead BOLTs were perceived as more >> "consensus critical", not subject to change, and more people were >> strongly encouraged to write specs for new lightning features >> elsewhere (like the BIP repo) then you would see this issue of growing >> BOLTs resolved. >> >> Cheers >> Ariel Lorenzo-Luaces >> >> On Wed, Jun 30, 2021 at 1:16 PM Olaoluwa Osuntokun wrote: >> > >> > > That being said I think all the points that are addressed in Ryan's mail >> > > could very well be formalized into BOLTs but maybe we just need to >> > > rethink >> > > the current process of the BOLTs to make it more accessible for new ideas >> > > to find their way into the BOLTs? >> > >> > I think part of what bLIPs are trying to solve here is promoting more >> > loosely >> > coupled evolution of the network. I think the BOLTs do a good job >> > currently of >> > specifying what _base_ functionality is required for a routing node in a >> > prescriptive manner (you must forward an HTLC like this, etc). However >> > there's >> > a rather large gap in describing functionality that has emerged over time >> > due >> > to progressive evolution, and aren't absolutely necessary, but enhance >> > node/wallet operation. >> > >> > Examples of include things like: path finding heuristics (BOLTs just say >
Re: [Lightning-dev] bLIPs: A proposal for community-driven app layer and protocol extension standardization
BIPs are already the Bazaar style of evolution that simultaneously allows flexibility and coordination/interoperability (since anyone can create a BIP and they create an environment of discussion). BOLTs are essentially one big BIP in the sense that they started as a place for discussion but are now more rigid. BOLTs must be followed strictly to ensure a node is interoperable with the network. And BOLTs should be rigid, as rigid as any widely used BIP like 32 for example. Even though BOLTs were flexible when being drafted their purpose has changed from descriptive to prescriptive. Any alternatives, or optional features should be extracted out of BOLTs, written as BIPs. The BIP should then reference the BOLT and the required flags set, messages sent, or alterations made to signal that the BIP's feature is enabled. A BOLT may at some point organically change to reference a BIP. For example if a BIP was drafted as an optional feature but then becomes more widespread and then turns out to be crucial for the proper operation of the network then a BOLT can be changed to just reference the BIP as mandatory. There isn't anything wrong with this. All of the above would work exactly the same if there was a bLIP repository instead. I don't see the value in having both bLIPs and BIPs since AFAICT they seem to be functionally equivalent and BIPs are not restricted to exclude lightning, and never have been. I believe the reason this move to BIPs hasn't happened organically is because many still perceive the BOLTs available for editing, so changes continue to be made. If instead BOLTs were perceived as more "consensus critical", not subject to change, and more people were strongly encouraged to write specs for new lightning features elsewhere (like the BIP repo) then you would see this issue of growing BOLTs resolved. Cheers Ariel Lorenzo-Luaces On Wed, Jun 30, 2021 at 1:16 PM Olaoluwa Osuntokun wrote: > > > That being said I think all the points that are addressed in Ryan's mail > > could very well be formalized into BOLTs but maybe we just need to rethink > > the current process of the BOLTs to make it more accessible for new ideas > > to find their way into the BOLTs? > > I think part of what bLIPs are trying to solve here is promoting more loosely > coupled evolution of the network. I think the BOLTs do a good job currently of > specifying what _base_ functionality is required for a routing node in a > prescriptive manner (you must forward an HTLC like this, etc). However there's > a rather large gap in describing functionality that has emerged over time due > to progressive evolution, and aren't absolutely necessary, but enhance > node/wallet operation. > > Examples of include things like: path finding heuristics (BOLTs just say you > should get from Alice to Bob, but provides no recommendations w.r.t _how_ to > do > so), fee bumping heuristics, breach retribution handling, channel management, > rebalancing, custom records usage (like the podcast index meta-data, > messaging, > etc), JIT channel opening, hosted channels, randomized channel IDs, fee > optimization, initial channel boostrapping, etc. > > All these examples are effectively optional as they aren't required for base > node operation, but they've organically evolved over time as node > implementations and wallet seek to solve UX and operational problems for > their users. bLIPs can be a _descriptive_ (this is how things can be done) > home for these types of standards, while BOLTs can be reserved for > _prescriptive_ measures (an HTLC looks like this, etc). > > The protocol as implemented today has a number of extensions (TLVs, message > types, feature bits, etc) that allow implementations to spin out their own > sub-protocols, many of which won't be considered absolutely necessary for node > operation. IMO we should embrace more of a "bazaar" style of evolution, and > acknowledge that loosely coupled evolution allows participants to more broadly > explore the design space, without the constraints of "it isn't a thing until N > of us start to do it". > > Historically, BOLTs have also had a rather monolithic structure. We've used > the same 11 or so documents for the past few years with the size of the > documents swelling over time with new exceptions, features, requirements, > etc. If you were hired to work on a new codebase and saw that everything is > defined in 11 "functions" that have been growing linearly over time, you'd > probably declare the codebase as being unmaintainable. By having distinct > documents for proposals/standards, bLIPs (author documents really), each new > standard/proposal is able to be more effectively explained, motivated, > versionsed, > etc. > > -- Laolu > > > On Wed, Jun 30, 2021 at 7:35 AM René Pickhardt via Lightning-dev > wrote: >> >> Hey everyone, >> >> just for reference when I was new here (and did not understand the processes >> well enough) I proposed a similar idea (called LIP) in 2018 c.f.: >> https
Re: [Lightning-dev] Routemap scaling (was: Just in Time Routing (JIT-Routing) and a channel rebalancing heuristic as an add on for improved routing success in BOLT 1.0)
> I am forking this thread as my reply is not at all related to the JIT-Routing. Sorry I think my last reply was also getting off subject as well. Thank you for forking the thread > Nonexistent channels (i.e. channels that do not have some backing UTXO in the > Bitcoin blockchain) are not safe to propagate on the network since they are > trivially spammable (i.e. can generate a large number of fake channels to > waste network bandwidth). I hadn't considered the effect this gossip would have on the network. Definitely nodes should not trust one another to tell the truth about nonexistent channels. The source node could blindly allow intermediate nodes to create sub-routes to the next hop. But I wouldn't favor this blind routing idea because fee calculations would be a complete guesses. > A good pruning heuristic is "channel capacity", which can be checked onchain > (the value of the UTXO backing the channel is the channel capacity). > It is good to keep channels with large capacity in the routemap, because such > large channels are more likely to successfully route a payment than smaller > channels. > So it is reasonable to delete channels with low capacity when the routemap > memory is becoming close to full. I'm generally concerned about these heuristics because any time nodes can game a heuristic they will be expected to do so. Gaming a heuristic can be good or bad depending on the side-effects. One side effect of the "channel capacity" heuristic is more reliable payments but another side effect is that low capacity (but possibly reliable) channels are neglected. > It seems to me that s/aggregate-channel/high-capacity-channel/g in your idea > would still work. > In effect, the high-capacity channel is very likely to successfully route in > either direction. > But if by chance it falls into a state where it is unable to route in one > direction or other, the intermediate node has other, lower-capacity channels > that it can use JIT-Routing with to improve the directionality of the > high-capacity channel. > Nothing in the JIT-Routing idea requires that the rebalancing loop is small, > only that a loop exists. > > Nodes still need to track their direct channels (so they are implicitly > always in the routemap). > But payee nodes making BOLT1 invoices could also provide `r` routes in the > invoice, with the given routes being from nodes with high-capacity channels, > so that even if the intermediate channels are pruned due to low capacity, it > is possible to get paid. Without low capacity channels spontaneous payments would not work. Or they would depend on asking for an invoice under the hood. It feels to me that at some point, someone with complete knowledge of the network needs to be employed. Cheers Ariel Lorenzo-Luaces ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Routemap scaling (was: Just in Time Routing (JIT-Routing) and a channel rebalancing heuristic as an add on for improved routing success in BOLT 1.0)
> I am forking this thread as my reply is not at all related to the JIT-Routing. Sorry I think my last reply was also getting off subject as well. Thank you for forking the thread > Nonexistent channels (i.e. channels that do not have some backing UTXO in the > Bitcoin blockchain) are not safe to propagate on the network since they are > trivially spammable (i.e. can generate a large number of fake channels to > waste network bandwidth). I hadn't considered the effect this gossip would have on the network. Definitely nodes should not trust one another to tell the truth about nonexistent channels. The source node could blindly allow intermediate nodes to create sub-routes to the next hop. But I wouldn't favor this blind routing idea because fee calculations would be a complete guess. > A good pruning heuristic is "channel capacity", which can be checked onchain > (the value of the UTXO backing the channel is the channel capacity). > It is good to keep channels with large capacity in the routemap, because such > large channels are more likely to successfully route a payment than smaller > channels. > So it is reasonable to delete channels with low capacity when the routemap > memory is becoming close to full. I'm generally concerned about these heuristics because any time nodes can game a heuristic they will be expected to do so. Gaming a heuristic can be good or bad depending on the side-effects. One side effect of the "channel capacity" heuristic is more reliable payments but another side effect is that low capacity (but possibly reliable) channels are neglected > It seems to me that s/aggregate-channel/high-capacity-channel/g in your idea > would still work. > In effect, the high-capacity channel is very likely to successfully route in > either direction. > But if by chance it falls into a state where it is unable to route in one > direction or other, the intermediate node has other, lower-capacity channels > that it can use JIT-Routing with to improve the directionality of the > high-capacity channel. > Nothing in the JIT-Routing idea requires that the rebalancing loop is small, > only that a loop exists. > > Nodes still need to track their direct channels (so they are implicitly > always in the routemap). > But payee nodes making BOLT1 invoices could also provide `r` routes in the > invoice, with the given routes being from nodes with high-capacity channels, > so that even if the intermediate channels are pruned due to low capacity, it > is possible to get paid. Without low capacity channels spontaneous payments would not work. Or they would depend on asking for an invoice under the hood. It feels to me that at some point, someone with complete knowledge of the network needs to eb employed. Cheers Ariel Lorenzo-Luaces ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
