[Lightning-dev] On the recent softforks survey, forget to fulfill my answer!
Hi, I was super glad to see the recent survey on potential softforks for the near-future of Bitcoin! I didn't have time to answer this one but will do so for the future. I wanna to salute the grassroots involvement in bitcoin protocol development, that's cool to see :) Though softforks are what shine in the media and social networks, one should not ignore they represent the aggregation of thousands of hours of sweat from contributors all across the ecosystem with discussion extending from IRC public or private chans, mailing list, medias, etc. What makes softfork discussion especially hard is that no one is following all those communications channels to collect the trace of information and as such it can be hard to reason on the Big Picture(tm). That's why soft-forks take time, and we might somehow be prepared for them to take even more time in the future... That said, where I would like to draw awareness of the community is about the submerged part of bitcoin protocol development iceberg. Softforks are sexy, though you have far more areas of Bitcoin dev who would benefit from a gentle boost by happy hands :p For e.g, if you take Bitcoin Core, you have few ongoing projects were folks have a hard time moving forward, e.g assumeutxo/mempool refactos/addr-relay/rebroadcasting module/mutation testing/ multiprocess/wallet external signer/GUI maintenance/libbitcoin_kernel[0] Those projects start to be "softfork"-in-itself-size-of-engineering, and for a lot of them might require more than pure "coding" skills, such as specification, simulations, extensive code coverage, up-to-date meeting documents. See what is currently done with the Core wiki [1] All those projects are modifying critical areas of Bitcoin such as the validation engine or the p2p stack and AFAICT, they deserve more care. Hopefully, by drawing the light there, more folks are going to understand them, we'll have more skilled reviewers, reducing the reliance on a few segments of the codebase being only understood by some seen experts and ideally, ingenious, "Many Eyes Make All Bugs Shallow" :) That said, it's only the technical ground and I believe the human layer of Bitcoin dev might be the one where grassroots-involvement might be the most fruitful. I would say the Bitcoin dev stage has changed a bit since the last 18 months, especially w.r.t to few factors, the arrival of massive development funding, the sudden mediatisation of protocol developers and the pursued geographical spreading, diversification and education of the poolset of contributors. When I did arrive on the stage a few years ago, funding was still a hard question, even for well-known, long-term contributors and only a few actors were taking care of Bitcoin. Really differently, from what we have seen on the last months, where we have seen a plethora of new organisations entering the game and benefiting from the generosity of the Bitcoin industry [2] Things have been so fast that sometimes one can wonder if there isn't a bubble around Bitcoin dev ? Few OGs might suggest we're back to 2017, with ICO-like webpage pinning "developers-as-brands". In reality, we see new grant announcements every month or week, but still the number of reviewers on Core doesn't seem to increase ? [3] Hopefully, a lot of those new structures pretending to work for Bitcoin betterness will get out of their childerness phase and slowly mature to something as sound as Chaincode or Square Crypto. Small, friendly, politics-free engineering teams with years-long stability, solving bitcoin problems with a "forever" perspective mindset. Though, as of today, you do have the opposite with the grant model. Being funded on the rational that yours peers "appreciate" your work is more going to generate implicit compliance at review time where you should instead spot their errors. Bitcoin development process is highly contrarian per nature, and constantly challenging your peers assumptions has been preserving software robustness. Time will separate the wheat from the chaff though how to make things better in the short term ? I don't know, maybe those structures could be exemplary and outsource their grant allocation decisions framework ? Or ask them to publish grant contract under which contributors are engaging themselves to observe if the usual independence provisions are present [4] In another direction, I believe the ongoing mediatization increase of the Bitcoin dev stage in the last months or so didn't improve the current state of affairs. We now see technical proposals, of which the soundness have not been thoroughly discussed in the traditional venues, being announced in big pump as some kind of "done-deal", potentially sustaining the false belief it has been already blessed or approved by the rest of the development community. And honestly, it's quite easy to approach any Bitcoin media today once you're a bit technical, and rely on lingo to create a perception of competency towards your interlocuto
Re: [Lightning-dev] Waiting SIGHASH_ANYPREVOUT and Packing Packages
I don't want to divert from the topic of this thread ("Waiting
SIGHASH_ANYPREVOUT and Packing Packages"), we can set up a separate
thread if we want to discuss this further. But just a couple of
things.
> Browsing quickly through Greg's piece, a lot of the reasoning is based on
> FOSS experience from Linux/Juniper, which to the best of my knowledge are
> centralized software projects ?
That is Greg's point. If Linux doesn't look further than the current
version and the next version with a BDFL (Linus) a decentralized
project like Bitcoin Core is going to struggle even more with longer
term roadmaps.
I think it is important to discuss what order changes should be
attempted but I agree with David that putting specific future version
numbers on changes is speculative at best and misleading at worst. The
record of previous predictions of what will be included in particular
future versions is not strong :)
> What was making sense when you had like ~20 Bitcoin dev with 90% of the
> technical knowledge doesn't scale when you have multiple second-layers
> specifications
It is great that we have a larger set of contributors in the ecosystem
today than back in say pre 2017. But today that set of contributors is
spread widely across a number of different projects that didn't exist
pre 2017. Changes to Core are (generally) likely to be implemented and
reviewed by current Core contributors as Lightning implementation
developers (generally) seem to have their hands full with their own
implementations.
I think we can get the balance right by making progress on this
(important) discussion whilst also maintaining humility that we don't
know exact timelines and that getting things merged into Core relies
on a number of people who have varying levels of interest and
understanding of L2 protocols.
On Mon, Jun 21, 2021 at 9:13 AM Antoine Riard wrote:
>
> Hi Dave,
>
> > That might work for current LN-penalty, but I'm not sure it works for
> eltoo.
>
> Well, we have not settled yet on the eltoo design but if we take the later
> proposal in date [0], signing the update transaction with SIGHGASH_ANYPREVOUT
> lets you attach non-interactively a single-party controlled input at
> broadcast-time. Providing the input amount is high enough to bump the
> transaction feerate over network mempools, it should allow the tx to
> propagate across network mempools and that way solve the pre-signed feerate
> problem as defined in the post ?
>
> > If Bitcoin Core can rewrite the blind CPFP fee bump transaction
> > to refer to any prevout, that implies anyone else can do the same.
> > Miners who were aware of two or more states from an eltoo channel would
> > be incentivized to rewrite to the oldest state, giving them fee revenue
> > now and ensuring fee revenue in the future when a later state update is
> > broadcast.
>
> Yep, you can add a per-participant key to lockdown the transaction and avoid
> any in-flight malleability ? I think this is discussed in the "A Stroll
> through Fee-Bumping Techniques" thread.
>
> > If the attacker using pinning is able to reuse their attack at no cost,
> > they can re-pin the channel again and force the honest user to pay
> > another anyprevout bounty to miners.
>
> This is also true with package-relay where your counterparty, with a better
> knowledge of network mempools, can always re-broadcast a CPFP-bumped
> malicious package ? Under this assumption, I think you should always be ready
> to bump our honest package.
>
> Further, for the clarity of the discussion, can you point to which pinning
> scenario you're thinking of or if it's new under SIGHASH_ANYPREVOUT, describe
> it ?
>
> > Repeat this a bunch of times and the honest user has now spent more on fees
> > than their balance from the
> closed channel.
>
> And sadly, as this concern also exists in case of a miner-harvesting attack
> against LN nodes, a concern that Gleb and I expressed more than a year ago in
> a public post [1], a good L2 client should always upper bound its fee-bumping
> reserve. I've a short though-unclear note on this notion of fee-bumping upper
> to warn other L2 engineers in "On Mempool Funny Games against Multi-Party
> Funded Transactions"
>
> Please read so:
>
> "A L2 client, with only a view of its mempool at best, won't understand why
> the transaction doesn't confirm and if it's responsible for the
> fee-bumping, it might do multiple rounds of feerate increase through CPFP,
> in vain. As the fee-bumping algorithm is assumed to be known if the victim
> client is open source code, the attacker can predict when the fee-bumping
> logic reaches its upper bound."
>
> Though thanks for the recall! I should log dynamic-balances in RL's
> `ChannelMonitorUpdate` for our ongoing implementation of anchor, updating my
> TODO :p
>
> > Even if my analysis above is wrong, I would encourage you or Matt or
> someone to write up this anyprevout idea in more detail and distribute
> it before you promote i
Re: [Lightning-dev] Waiting SIGHASH_ANYPREVOUT and Packing Packages
Hi Dave,
> That might work for current LN-penalty, but I'm not sure it works for
eltoo.
Well, we have not settled yet on the eltoo design but if we take the later
proposal in date [0], signing the update transaction with
SIGHGASH_ANYPREVOUT lets you attach non-interactively a single-party
controlled input at broadcast-time. Providing the input amount is high
enough to bump the transaction feerate over network mempools, it should
allow the tx to propagate across network mempools and that way solve the
pre-signed feerate problem as defined in the post ?
> If Bitcoin Core can rewrite the blind CPFP fee bump transaction
> to refer to any prevout, that implies anyone else can do the same.
> Miners who were aware of two or more states from an eltoo channel would
> be incentivized to rewrite to the oldest state, giving them fee revenue
> now and ensuring fee revenue in the future when a later state update is
> broadcast.
Yep, you can add a per-participant key to lockdown the transaction and
avoid any in-flight malleability ? I think this is discussed in the "A
Stroll through Fee-Bumping Techniques" thread.
> If the attacker using pinning is able to reuse their attack at no cost,
> they can re-pin the channel again and force the honest user to pay
> another anyprevout bounty to miners.
This is also true with package-relay where your counterparty, with a better
knowledge of network mempools, can always re-broadcast a CPFP-bumped
malicious package ? Under this assumption, I think you should always be
ready to bump our honest package.
Further, for the clarity of the discussion, can you point to which pinning
scenario you're thinking of or if it's new under SIGHASH_ANYPREVOUT,
describe it ?
> Repeat this a bunch of times and the honest user has now spent more on
fees than their balance from the
closed channel.
And sadly, as this concern also exists in case of a miner-harvesting attack
against LN nodes, a concern that Gleb and I expressed more than a year ago
in a public post [1], a good L2 client should always upper bound its
fee-bumping reserve. I've a short though-unclear note on this notion of
fee-bumping upper to warn other L2 engineers in "On Mempool Funny Games
against Multi-Party Funded Transactions"
Please read so:
"A L2 client, with only a view of its mempool at best, won't understand why
the transaction doesn't confirm and if it's responsible for the
fee-bumping, it might do multiple rounds of feerate increase through CPFP,
in vain. As the fee-bumping algorithm is assumed to be known if the victim
client is open source code, the attacker can predict when the fee-bumping
logic reaches its upper bound."
Though thanks for the recall! I should log dynamic-balances in RL's
`ChannelMonitorUpdate` for our ongoing implementation of anchor, updating
my TODO :p
> Even if my analysis above is wrong, I would encourage you or Matt or
someone to write up this anyprevout idea in more detail and distribute
it before you promote it much more.
That's a really fair point, as a lot of the reasoning was based on private
discussion with Matt. Though as SIGHASH_ANYPREVOUT isn't advocated for
community consensus and those things take time, should just take a few
hours of my time.
> Even if every protocol based on presigned transactions can magically
allow dynamically adding inputs and modifying outputs for fees, and we
also have a magic perfect transaction replacement protocol,
"“Any sufficiently advanced technology is indistinguishable from magic.”
Arthur C. Clarke
Wit apart, that might be the outcome with careful bitcoin protocol
development, where technical issues are laid out in a best effort (of
mine!) and spread to the Bitcoin community on the most public bitcoin
communication channel ?
And humbly, on all those L2 issues I did change my opinion, as I've written
so much explicitly in this thread post by pointing to an older post of mine
("Advances in Bitcoin Contracting : Uniform Policy and Package Relay").
This reversal, partially motivated by a lot of discussion with folks,
including yourself, initiated since at least mid last year.
> package relay is still fundamentally useful for CPFP fee bumping very low
> feerate transactions received from an external party. E.g. Alice pays
> Bob, mempool min feerates increase and Alice's transaction is dropped,
> Bob still wants the money, so he submits a package with Alice's
> transaction plus his own high feerate spend of it.
I think this point would be a reverse of our p2p design where we are now
making the sender responsible for the receiver quality of its mempool
feerate ? This question has never been clear up during the years-long
discussion of package-relay design [1].
Though referring to the thread post and last week's transaction-relay
workshop, I did point out that package-relay might serve in the long-term
as a mempool-sync mechanism to prevent potential malicious mempool
partitions [2].
> Package relay is a clear improvement now, and one I expec
