Re: [Lightning-dev] bLIPs: A proposal for community-driven app layer and protocol extension stand

2021-07-02 Thread fiatjaf
If the BIPs can allow very small standards related to a small niche of
Lightning usage, then I think they are the place for everything indeed. I'm
convinced.

Thinking about proposing the LNURL specs as BIPs now, but then I don't know
if it will be weird for them to exist alone there, without the basis of the
lightning technology to support them. I hope the BOLT authors investigate
moving them there too so the other Lightning BIPs can add the BOLT BIPs as
"Required".

The only question to me is this: should each BOLT be a BIP? Or all BOLTs be
mashed together as a single BIP? Then what happens when Taproot-based
channels, PTLCs or Eltoo-based channels get implemented? They are added as
new BIPs that inherit and modify the previous?

I also went through
https://github.com/bitcoin/bips/wiki/BIP-Process-wishlist and to me they
seem to be all good proposals (specially the list of projects/services
implementing the BIP). Except BIP versioning. I like that BIPs have
meaningless numbers, just add another BIP and refer to it by a number. For
that reason I also don't like prepending an "L" to Lightning-related BIPs
(more so because some of these may be reused later in non-Lightning
contexts, who knows?). Anyway I'm fine with anything.

On Fri, Jul 2, 2021 at 4:32 PM Luke Dashjr  wrote:

> Yes, many systems doesn't really make sense. We can add editors and revise
> the
> BIP process as needed (BOLTs might prefer to use markdown?). Even aside
> from
> Lightning BIPs, there are several improvements that can be made, so it
> makes
> sense to address everything at once.
>
> https://github.com/bitcoin/bips/wiki/BIP-Process-wishlist
>
> The BIP 2xx range has been set aside for Lightning years ago, and we can
> do
> similar things to help keep things organized within BIPs. Kalle suggested
> maybe it'd be better to do BIP "L###" instead, and perhaps that would work
> better if there's likely to be several sub-namespaces.
>
> Luke
>
>
> On Friday 02 July 2021 12:02:28 Dan Gershony wrote:
> > Hi,
> > There will be many layer 2 (and probably layer 3)  protocols (BOLT, RGB,
> > Volts etc...) does it really make sense to merge them all into the BIPs
> > system?
> >
> >
> > On Fri, Jul 2, 2021 at 10:03 AM nathanael via Lightning-dev <
> >
> > lightning-dev@lists.linuxfoundation.org> wrote:
> > > Michael Folkson  wrote:
> > > > > Adding a third BIP editor more involved with Lightning sounds like
> a
> > >
> > > good idea.
> > >
> > > > Or alternatively if BOLTs were subsumed into BIPs I think Bastien
> > > > would be a great additional BIP editor to cover Lightning related
> BIPs
> > > >
> > > > :) I think BOLTs being subsumed into BIPs would be nice but I'm
> > > >
> > > > pessimistic it will happen. Like legislation and regulation in the
> > > > legacy financial system alphabet soups only expand they never get
> > > > simplified. Let's at least resist alphabet soup expansion here.
> > >
> > > arent lightning improvements in the end bitcoin improvements too?
> > > i am thinking of bips like the rfcs of the internet
> > >
> > > --
> > > nathanael
> > > ___
> > > Lightning-dev mailing list
> > > Lightning-dev@lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] bLIPs: A proposal for community-driven app layer and protocol extension stand

2021-07-02 Thread Luke Dashjr
Yes, many systems doesn't really make sense. We can add editors and revise the 
BIP process as needed (BOLTs might prefer to use markdown?). Even aside from 
Lightning BIPs, there are several improvements that can be made, so it makes 
sense to address everything at once.

https://github.com/bitcoin/bips/wiki/BIP-Process-wishlist

The BIP 2xx range has been set aside for Lightning years ago, and we can do 
similar things to help keep things organized within BIPs. Kalle suggested 
maybe it'd be better to do BIP "L###" instead, and perhaps that would work 
better if there's likely to be several sub-namespaces.

Luke


On Friday 02 July 2021 12:02:28 Dan Gershony wrote:
> Hi,
> There will be many layer 2 (and probably layer 3)  protocols (BOLT, RGB,
> Volts etc...) does it really make sense to merge them all into the BIPs
> system?
>
>
> On Fri, Jul 2, 2021 at 10:03 AM nathanael via Lightning-dev <
>
> lightning-dev@lists.linuxfoundation.org> wrote:
> > Michael Folkson  wrote:
> > > > Adding a third BIP editor more involved with Lightning sounds like a
> >
> > good idea.
> >
> > > Or alternatively if BOLTs were subsumed into BIPs I think Bastien
> > > would be a great additional BIP editor to cover Lightning related BIPs
> > >
> > > :) I think BOLTs being subsumed into BIPs would be nice but I'm
> > >
> > > pessimistic it will happen. Like legislation and regulation in the
> > > legacy financial system alphabet soups only expand they never get
> > > simplified. Let's at least resist alphabet soup expansion here.
> >
> > arent lightning improvements in the end bitcoin improvements too?
> > i am thinking of bips like the rfcs of the internet
> >
> > --
> > nathanael
> > ___
> > Lightning-dev mailing list
> > Lightning-dev@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] bLIPs: A proposal for community-driven app layer and protocol extension standardization

2021-07-02 Thread Antoine Riard
Hi Ryan,

Thanks for starting this discussion, I agree it's a good time for the
Lightning development community to start this self-introspection on its own
specification process :)

First and foremost, maybe we could take a minute off to celebrate the
success of the BOLT process and the road traveled so far ? What was a fuzzy
heap of ideas on a whiteboard a few years ago has bloomed up to a living
and pulsating distributed ecosystem of thousands of nodes all around the
world. If the bet was to deliver on fast, instant, cheap, reasonably
scalable, reasonably confidential Bitcoin payments, it's a won one and
that's really cool.

Retrospectively, it was a foolhardy bet for a wide diversity of factors.
One could think about opinionated, early design choices deeply affecting
protocol safety and efficiency of which the ultimate validity was still a
function of fluky base layer evolutions [0]. Another could consider the
communication challenges of softly aligning development teams on the common
effort of designing and deploying from scratch a cryptographic protocol as
sophisticated as Lightning. Not an easy task when you're mindful about the
timezones spread, the diversity of software engineering backgrounds and the
differing schedules of priorities.

So kudos to everyone who has played a part in the Lightning dev process.
The OGs who started the tale, the rookies who jumped on the wagon on the
way and today newcomers showing up with new seeds to nurture the ecosystem
:)

Now, I would say we more-or-less all agree that the current BOLT process
has reached its limits. Both from private conservations across the teams
but also frustrations expressed during the irc meetings in the past months.
Or as a simple data point, the only meaningful spec object we did merge on
the last 18 months is anchor output, it did consumes a lot of review and
engineering bandwidth from contributors, took few refinement to finalize
(`option_anchors_zero_fee_htlc_tx`) and I believe every implementations are
still scratching their heads on a robust, default fee-bumping strategy.

So if we agree about the BOLT process limitations, the next question to
raise is how to improve it. Though there, as expressed in other replies,
I'm more we're not going to be able to do that much, as ultimately we're
upper bounded by a fast-pacing, always-growing, permissionless ecosystem of
applications and experiments moving forward in baazar-style and
lower-bounded by a decentralized process across teams allocating their
engineering resources with different priorities or even exploring Lightning
massive evolution stages in heterogenous, synergic directions.

Breeding another specification process on top of Lightning sounds a good
way forward. Though I believe it might be better to take time to operate
the disentanglement nicely. If we take the list of ideas which could be
part of such a process, one of them, dynamic commitments could make a lot
of sense to be well-designed and well-supported by every implementation. In
case of emergency fixes to deploy safer channel types, if you have to close
all your channels with other implementations, on a holistic scale, it might
cloak the mempools and spike the feerate, strickening safety of every other
channel on the network. Yes we might have safety interdepencies between
implementations :/

And it's also good to have thoughtful, well-defined specification bounds
when you're working on coordinated security disclosures to know who has
implemented what and whom you should reach out when something is broken.

Another orthogonal point to consider is the existence of already
higher-layer protocol specifications such as the dlcspecs. Even if the
ecosystem is still in the bootstrap phase for now, we already have a
discussion to split between a "consensus" track and more optional features.
I believe some features discussed there such as negotiation layer about
premium fee to compensate unilateral fee-bumping responsibility risk could
belong to such a new bLIPs process ?

So here my thinking, as a BOLT contributor, what the common subset of
problems we want to keep tackling down together in the coming years, what
is the remaining subset we're happy to be engage by a higher layer
development community and how to draw both communication and software
interfaces in-between ?

Personally, I would be glad if we not extend the scope of the current BOLT
coverage and focus more on fixing the known-issues, simplifying state
machines, fixing oddities of channel policies announcements [1], writing
down best practices on fee-bumping strategies, agreeing on channel types
upgrades raw mechanisms, features discovery and if we want to innovate
focus on taproot well-done integration which should keep us busy for few
years, among others PTLC support, funding output taproot support,
composable taptree for revokeable outputs, ...

IHMO, if the BOLT process is officialized it will enter in a more boring
phase, focused on safety/reliability/privacy 

Re: [Lightning-dev] bLIPs: A proposal for community-driven app layer and protocol extension stand

2021-07-02 Thread Dan Gershony
Hi,
There will be many layer 2 (and probably layer 3)  protocols (BOLT, RGB,
Volts etc...) does it really make sense to merge them all into the BIPs
system?


On Fri, Jul 2, 2021 at 10:03 AM nathanael via Lightning-dev <
lightning-dev@lists.linuxfoundation.org> wrote:

> Michael Folkson  wrote:
> > > Adding a third BIP editor more involved with Lightning sounds like a
> good idea.
> >
> > Or alternatively if BOLTs were subsumed into BIPs I think Bastien
> > would be a great additional BIP editor to cover Lightning related BIPs
> > :) I think BOLTs being subsumed into BIPs would be nice but I'm
> > pessimistic it will happen. Like legislation and regulation in the
> > legacy financial system alphabet soups only expand they never get
> > simplified. Let's at least resist alphabet soup expansion here.
>
> arent lightning improvements in the end bitcoin improvements too?
> i am thinking of bips like the rfcs of the internet
>
> --
> nathanael
> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] bLIPs: A proposal for community-driven app layer and protocol extension stand

2021-07-02 Thread nathanael via Lightning-dev
Michael Folkson  wrote:
> > Adding a third BIP editor more involved with Lightning sounds like a good 
> > idea.
> 
> Or alternatively if BOLTs were subsumed into BIPs I think Bastien
> would be a great additional BIP editor to cover Lightning related BIPs
> :) I think BOLTs being subsumed into BIPs would be nice but I'm
> pessimistic it will happen. Like legislation and regulation in the
> legacy financial system alphabet soups only expand they never get
> simplified. Let's at least resist alphabet soup expansion here.

arent lightning improvements in the end bitcoin improvements too?
i am thinking of bips like the rfcs of the internet

--
nathanael
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev


Re: [Lightning-dev] bLIPs: A proposal for community-driven app layer and protocol extension standardization

2021-07-02 Thread Michael Folkson
> The other thing bLIPs do is do away with the whole "human picks the number of 
> documents", and "don't assign your own number, you must wait".

So TL;DR BIPs and BOLTs sometimes require waiting for things (like
review and consensus) and there should be a new acronym and process
("bLIPs") to avoid us having to wait for things. I just think "bLIPs"
adds confusion e.g. should something be a bLIP or a BOLT? Does a bLIP
eventually become a BOLT when it is mature enough? This tendency to
fragment and introduce new acronyms and new processes should be
resisted imo. If a new process is introduced every time there is a
disagreement or perceived friction it just erodes the value of
existing processes and means they all get bypassed. Strengthen and
improve existing processes and only introduce a new one as an absolute
last resort.

Other than the minor frictions described above I don't see why "bLIPs"
can't just be draft BOLTs.

> Adding a third BIP editor more involved with Lightning sounds like a good 
> idea.

Or alternatively if BOLTs were subsumed into BIPs I think Bastien
would be a great additional BIP editor to cover Lightning related BIPs
:) I think BOLTs being subsumed into BIPs would be nice but I'm
pessimistic it will happen. Like legislation and regulation in the
legacy financial system alphabet soups only expand they never get
simplified. Let's at least resist alphabet soup expansion here.

On Fri, Jul 2, 2021 at 9:01 AM Bastien TEINTURIER  wrote:
>>
>> Will it actually add any more fragmentation that already exists? Due to all
>> the extensibility we've added in the protocol, it's already possible for any
>> implementation to start to work on their own sub-protocols. This just gives
>> them a new venue to at least _describe_ what they're using.
>
>
> It's only my 2 cents, but I'm afraid it will indeed add more fragmentation, 
> because
> the fact that there exists a bLIP for feature XXX will likely act as a green 
> light to
> deploy it faster instead of spending more time talking about it with the 
> community
> and thinking about potential issues, forward-compatibility, etc.
>
> But I agree with you that it also gives more freedom to experiment in the 
> real world,
> which helps find issues and correct them, paving the way for better features 
> for
> end users.
>
>> It's also likely the case that already implementations, or typically forks
>> of implementations are already using "undocumented" TLVs or feature bits in
>> the wild today.
>
>
> But today we're usually very careful when we do that, and use numbers in high 
> ranges
> for these use-cases. In our case for example we use message type 35007 for our
> swap-in and we expect that to change once standardized, so we did extra work 
> to
> ensure we wouldn't paint ourselves into a corner when switching to a standard 
> version.
>
> I think that if we have a centralized bLIP repo, we can take this opportunity 
> to safely
> assign "final" values for types and feature bits that are used by each bLIP, 
> and stronger
> guarantees that they will not conflict with another bLIP or BOLT. Of course 
> that doesn't
> stop anyone from deploying a conflict, but their use of the same bits won't 
> be documented
> so it shouldn't be widely deployed, and browsing the BOLTs and bLIPs should 
> let anyone
> see what the "correct" meaning of those bits should be.
>
> Cheers,
> Bastien
>
>
> Le jeu. 1 juil. 2021 à 22:43, Olaoluwa Osuntokun  a écrit :
>>
>> > But they don't address the first point at all, they instead work around
>> > it.  To be fair, I don't think we can completely address that first point:
>> > properly reviewing spec proposals takes a lot of effort and accepting
>> > complex changes to the BOLTs shouldn't be done lightly.
>>
>> I think this is a fair characterization that I agree with. I also agree that
>> there isn't really a way to fundamentally address it. The issue of scarce
>> review resources is something just about any large open source project needs
>> to deal with: everyone wants to make a PR, but no one wants to review the
>> PRs of others, unless it scratches some tangential itch they may have. IMO
>> it's also the case that the problem/solution space of LN is so large, that
>> it's hard to expect every developer to review each new proposal that comes
>> in, as they themselves have their own set of priorities (product,
>> businesses, protocol, personal, etc).
>>
>> In the end though, I think when there've been critical items that affect all
>> implementations and/or the existence of the protocol itself, developers
>> typically band together to commit resources to help a proposal move forward.
>> One upcoming example of this will be the "base" taproot channel type (the
>> design space is pretty large in that it even permits a new type of symmetric
>> state revocation-based channel).
>>
>> >  it will add fragmentation to the network, it will add maintenance costs
>> >  and backwards-compatibility issues
>>
>> Will it actually add 

Re: [Lightning-dev] bLIPs: A proposal for community-driven app layer and protocol extension standardization

2021-07-02 Thread Bastien TEINTURIER
>
> Will it actually add any more fragmentation that already exists? Due to all
> the extensibility we've added in the protocol, it's already possible for
> any
> implementation to start to work on their own sub-protocols. This just gives
> them a new venue to at least _describe_ what they're using.
>

It's only my 2 cents, but I'm afraid it will indeed add more fragmentation,
because
the fact that there exists a bLIP for feature XXX will likely act as a
green light to
deploy it faster instead of spending more time talking about it with the
community
and thinking about potential issues, forward-compatibility, etc.

But I agree with you that it also gives more freedom to experiment in the
real world,
which helps find issues and correct them, paving the way for better
features for
end users.

It's also likely the case that already implementations, or typically forks
> of implementations are already using "undocumented" TLVs or feature bits in
> the wild today.
>

But today we're usually very careful when we do that, and use numbers in
high ranges
for these use-cases. In our case for example we use message type 35007 for
our
swap-in and we expect that to change once standardized, so we did extra
work to
ensure we wouldn't paint ourselves into a corner when switching to a
standard version.

I think that if we have a centralized bLIP repo, we can take this
opportunity to safely
assign "final" values for types and feature bits that are used by each
bLIP, and stronger
guarantees that they will not conflict with another bLIP or BOLT. Of course
that doesn't
stop anyone from deploying a conflict, but their use of the same bits won't
be documented
so it shouldn't be widely deployed, and browsing the BOLTs and bLIPs should
let anyone
see what the "correct" meaning of those bits should be.

Cheers,
Bastien


Le jeu. 1 juil. 2021 à 22:43, Olaoluwa Osuntokun  a
écrit :

> > But they don't address the first point at all, they instead work around
> > it.  To be fair, I don't think we can completely address that first
> point:
> > properly reviewing spec proposals takes a lot of effort and accepting
> > complex changes to the BOLTs shouldn't be done lightly.
>
> I think this is a fair characterization that I agree with. I also agree
> that
> there isn't really a way to fundamentally address it. The issue of scarce
> review resources is something just about any large open source project
> needs
> to deal with: everyone wants to make a PR, but no one wants to review the
> PRs of others, unless it scratches some tangential itch they may have. IMO
> it's also the case that the problem/solution space of LN is so large, that
> it's hard to expect every developer to review each new proposal that comes
> in, as they themselves have their own set of priorities (product,
> businesses, protocol, personal, etc).
>
> In the end though, I think when there've been critical items that affect
> all
> implementations and/or the existence of the protocol itself, developers
> typically band together to commit resources to help a proposal move
> forward.
> One upcoming example of this will be the "base" taproot channel type (the
> design space is pretty large in that it even permits a new type of
> symmetric
> state revocation-based channel).
>
> >  it will add fragmentation to the network, it will add maintenance costs
> >  and backwards-compatibility issues
>
> Will it actually add any more fragmentation that already exists? Due to all
> the extensibility we've added in the protocol, it's already possible for
> any
> implementation to start to work on their own sub-protocols. This just gives
> them a new venue to at least _describe_ what they're using. As usual, it's
> up to other implementations if they want to adopt it or not, or advise
> against its use.
>
> >  many bLIPs will be sub-optimal solutions to the problem they try to
> solve
> >  and some bLIPs will be simply insecure and may put users' funds at risk
> >  (L2 protocols are hard and have subtle issues that can be easily missed)
>
> This may be the case, but I guess at times it's hard to know if something
> is
> objectively sub-optimal without further exploration of the design space,
> which usually means either more people involved, or more time examining the
> problem. Ultimately, different wallets/implementations may also be willing
> to make different usability/security trade-offs. One example here is zero
> conf channels: they assume a greater degree of trust with the party you're
> _accepting_ the channel from, as if you receive funds over the channel,
> they
> can be double spent away. However it's undeniable that they improve the UX
> by reducing the amount of time a user needs to wait around before they can
> actually jump in and use LN.
>
> In the end though, there's no grand global committee that prevents people
> from deploying software they think is interesting or useful. In the long
> run, I guess one simply needs to hope that bad ideas die out, or speak out
>