Re: [Lightning-dev] Disclosure: Fake channel DoS vector

[Bastien TEINTURIER]

Hey Matt,

Great work on finding this issue, thoroughly testing it against
implementations,
and on the follow-up you did after reporting this to the various teams.

We all agree that having more people spending time poking at the code
to find issues is very beneficial for the project. I hope your work will
encourage
more people to work on similar research! Protecting p2p networks against
DoS vectors is a hard task, and the more eyes on the project, the better.

Thanks,
Bastien

Le lun. 28 août 2023 à 08:46, Antoine Riard  a
écrit :

> Hi Matt,
>
> > You've definitely done some review for some subset of code, mostly the
> anchors code which was added
> > not too long ago, but please don't pretend you've reviewed a large
> volume of the pull requests in
> > LDK, as far as I understand you have several other projects you focus
> heavily on, which is great,
> > but that's not being a major LDK contributor.
>
> This is insulting, as I remember very well reviewing some hard parts of
> recent ongoing changes such as some API changes for watchtower integration
> or the state machine for collaborative transaction construction. Adopting a
> pure quantitative measurement of the review contributions is short-sighted
> and I think your position forgets with any mature open-source project
> review of the hard and sensitive part of the codebase is the bottleneck.
>
> We're working on a decentralized bitcoin open-source project, there is no
> benevolent dictator for life with the legitimacy to qualify who is a major
> or "spokesperson" contributor, and who is not. I understand this is your
> job to work full-time on LDK, though I don't think there is any contractual
> provision in your work contract requesting to play the BDLF.
>
> If you have a different viewpoint or other professional information to
> communicate to the community, thanks.
>
> > In 2022 and 2023 you:
> >  * landed a PR removing yourself from the security-reporting list
> (#2323, no idea why you're trying
> > to speak for the project when you removed yourself!)
> >  * fixed one bug in the anchors aggregation stuff before it was released
> (#1841, thanks!)
> >  * made some constants public (#1839)
> >  * increase a constant (#1532)
> >  * added a trivial double-check of user code (#1531)
>
> If I remember very well my anchor output patchset was ready to land in the
> second-half of 2020, though at that time we didn't have enough qualified
> review bandwidth and I spent a hell of a lot of time reviewing other
> people's contributions through 2020/2021 to move the project nearer from
> production-readiness. That lesson about the anchor output patchset, i.e
> coming with meaningful code diff to do _interesting_ things and solve hard
> problems as quite rendered me frivolous to show up with big diff, and waste
> my coding time (when I can make advance on solving LN-related problems in
> Bitcoin Core).
>
> I think very recently I proposed changes to advance mempool monitoring and
> custom script support, though here again it sounds to me we're still
> lacking qualified eyes to bring informed technical opinions on the areas
> necessitating a lot of context.
>
> About #2323, the reason of removing myself from the security-reporting
> list is related to weak and non-consensual code of conduct you introduced
> last year which brings severe vulnerabilities to LDK process w.r.t social
> attacks by external actors and risks of long-term shitshow a la Rust
> governance, which I raised immediately on the code of conduct PR. I think
> you never answered me neither publicly or privately on the lack of
> robustness of this code of conduct if we're targeted by psyops from
> NK-hacking groups (Sadly, real-world thing in the cryptocurrency world).
>
> I have no doubt we'll be able to rebuild consensus on our
> security-handling / project community process in the future, with calm and
> patience.
>
> > You've also, to my knowledge, never joined the public bi-weekly LDK
> development calls, don't join
> > the lightning spec meeting, and don't engage in the public discord
> discussions where development
> > decisions are made.
>
> Again this is insulting to use the word "never" as I was the original host
> of the LDK development meetings on Slack and I think I took the initiative
> to launch the LDK review club last year. All those meetings / communication
> spaces have public logs to parse interesting to look on and in fine
> development decisions are made in a continuous fashion, where the review
> and testing process on the repository being the main factor. And I'm pretty
> active reviewing things on the lightning spec side at least as much as you.
>
> > This implies you absolutely don't have a deep understanding of all the
> things happening in the
> > project, which makes you poorly suited to speak on behalf of the
> project. I'm not trying to pass
> > judgement on whether you've contributed (you have! thanks for your
> contributions!), but only
> > suggesting that if you 

Re: [Lightning-dev] Disclosure: Fake channel DoS vector

[Antoine Riard]

Hi Matt,

> You've definitely done some review for some subset of code, mostly the
anchors code which was added
> not too long ago, but please don't pretend you've reviewed a large volume
of the pull requests in
> LDK, as far as I understand you have several other projects you focus
heavily on, which is great,
> but that's not being a major LDK contributor.

This is insulting, as I remember very well reviewing some hard parts of
recent ongoing changes such as some API changes for watchtower integration
or the state machine for collaborative transaction construction. Adopting a
pure quantitative measurement of the review contributions is short-sighted
and I think your position forgets with any mature open-source project
review of the hard and sensitive part of the codebase is the bottleneck.

We're working on a decentralized bitcoin open-source project, there is no
benevolent dictator for life with the legitimacy to qualify who is a major
or "spokesperson" contributor, and who is not. I understand this is your
job to work full-time on LDK, though I don't think there is any contractual
provision in your work contract requesting to play the BDLF.

If you have a different viewpoint or other professional information to
communicate to the community, thanks.

> In 2022 and 2023 you:
>  * landed a PR removing yourself from the security-reporting list (#2323,
no idea why you're trying
> to speak for the project when you removed yourself!)
>  * fixed one bug in the anchors aggregation stuff before it was released
(#1841, thanks!)
>  * made some constants public (#1839)
>  * increase a constant (#1532)
>  * added a trivial double-check of user code (#1531)

If I remember very well my anchor output patchset was ready to land in the
second-half of 2020, though at that time we didn't have enough qualified
review bandwidth and I spent a hell of a lot of time reviewing other
people's contributions through 2020/2021 to move the project nearer from
production-readiness. That lesson about the anchor output patchset, i.e
coming with meaningful code diff to do _interesting_ things and solve hard
problems as quite rendered me frivolous to show up with big diff, and waste
my coding time (when I can make advance on solving LN-related problems in
Bitcoin Core).

I think very recently I proposed changes to advance mempool monitoring and
custom script support, though here again it sounds to me we're still
lacking qualified eyes to bring informed technical opinions on the areas
necessitating a lot of context.

About #2323, the reason of removing myself from the security-reporting list
is related to weak and non-consensual code of conduct you introduced last
year which brings severe vulnerabilities to LDK process w.r.t social
attacks by external actors and risks of long-term shitshow a la Rust
governance, which I raised immediately on the code of conduct PR. I think
you never answered me neither publicly or privately on the lack of
robustness of this code of conduct if we're targeted by psyops from
NK-hacking groups (Sadly, real-world thing in the cryptocurrency world).

I have no doubt we'll be able to rebuild consensus on our security-handling
/ project community process in the future, with calm and patience.

> You've also, to my knowledge, never joined the public bi-weekly LDK
development calls, don't join
> the lightning spec meeting, and don't engage in the public discord
discussions where development
> decisions are made.

Again this is insulting to use the word "never" as I was the original host
of the LDK development meetings on Slack and I think I took the initiative
to launch the LDK review club last year. All those meetings / communication
spaces have public logs to parse interesting to look on and in fine
development decisions are made in a continuous fashion, where the review
and testing process on the repository being the main factor. And I'm pretty
active reviewing things on the lightning spec side at least as much as you.

> This implies you absolutely don't have a deep understanding of all the
things happening in the
> project, which makes you poorly suited to speak on behalf of the project.
I'm not trying to pass
> judgement on whether you've contributed (you have! thanks for your
contributions!), but only
> suggesting that if you don't contribute regularly enough to have a good
understanding of everything
> going on, speaking on behalf of the project isn't appropriate.

If you like to judge my "deep understanding" of LDK codebase, you're free
to throw a ldk-sample with the latest v0.0.116 release , throw your money
on it on few channels, send me the node public key and then you'll see
after a while if the funds are still there, otherwise this is cheap talk
and just the subjective appreciation of your mind.

In reality, I'm still weekly reviewing and hacking on Bitcoin Core's
sensitive parts related to Lightning, and I don't necessarily have the time
to attend *all* the meetings. As a LDK maintainer you should be 

Re: [Lightning-dev] Disclosure: Fake channel DoS vector

[Matt Corallo]

On 8/26/23 5:03 AM, Antoine Riard wrote:

Hi Matt,

 > While you were aware of these fixes at the time, I'd appreciate it if you, 
someone who hasn't spent
 > much time contributing to LDK over the past two or three years, stop trying 
to speak on behalf of
 > the LDK project.

While this statement is blatantly false and disregards all the review


You've definitely done some review for some subset of code, mostly the anchors code which was added 
not too long ago, but please don't pretend you've reviewed a large volume of the pull requests in 
LDK, as far as I understand you have several other projects you focus heavily on, which is great, 
but that's not being a major LDK contributor.


and robustness hardening 
landed during the last two or three years


In 2022 and 2023 you:
 * landed a PR removing yourself from the security-reporting list (#2323, no idea why you're trying 
to speak for the project when you removed yourself!)

 * fixed one bug in the anchors aggregation stuff before it was released 
(#1841, thanks!)
 * made some constants public (#1839)
 * increase a constant (#1532)
 * added a trivial double-check of user code (#1531)

You've also, to my knowledge, never joined the public bi-weekly LDK development calls, don't join 
the lightning spec meeting, and don't engage in the public discord discussions where development 
decisions are made.


This implies you absolutely don't have a deep understanding of all the things happening in the 
project, which makes you poorly suited to speak on behalf of the project. I'm not trying to pass 
judgement on whether you've contributed (you have! thanks for your contributions!), but only 
suggesting that if you don't contribute regularly enough to have a good understanding of everything 
going on, speaking on behalf of the project isn't appropriate.


I would appreciate it from you in the conduct of your 
maintenance janitorial role to have more regard for the LDK users funds security rather than a "move 
fast and break things" attitude.


While I know you feel like lightning at large isn't a protocol which takes security seriously, I 
think you're pretty far off base here. Getting lightning right is *hard*, as you well know there are 
many, many, many ways it can go wrong. And we, like every other lightning software project, take 
that seriously, while also trying to ship features to make lightning broadly useful and usable (two 
things that its historically not really been...because its hard for many reasons beyond just 
security issues).


If you followed LDK (and other lightning) development more closely, I think you'd have a greater 
appreciation for these things :).


As such, and with in mind all open-source ethical rules, I'll keep speaking on the behalf of the LDK 
project when I see fit, whether you're pleased or not.


I'm really unsure what you mean here "open-source ethical rules" - is it your opinion that you 
should speak for a project you don't really follow closely just because you think the people who do 
work on it a lot aren't doing a good enough job in your opinion? That seems incredibly strange to me.


Matt
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Disclosure: Fake channel DoS vector

[Antoine Riard]

Hi Matt,

> While you were aware of these fixes at the time, I'd appreciate it if
you, someone who hasn't spent
> much time contributing to LDK over the past two or three years, stop
trying to speak on behalf of
> the LDK project.

While this statement is blatantly false and disregards all the review and
robustness hardening landed during the last two or three years, I would
appreciate it from you in the conduct of your maintenance janitorial role
to have more regard for the LDK users funds security rather than a "move
fast and break things" attitude.

As such, and with in mind all open-source ethical rules, I'll keep speaking
on the behalf of the LDK project when I see fit, whether you're pleased or
not.

Best,
Antoine

Le ven. 25 août 2023 à 20:30, Matt Corallo  a
écrit :

> While you were aware of these fixes at the time, I'd appreciate it if you,
> someone who hasn't spent
> much time contributing to LDK over the past two or three years, stop
> trying to speak on behalf of
> the LDK project.
>
> Thanks,
> Matt
>
> On 8/24/23 4:33 PM, Antoine Riard wrote:
> > Hi Matt,
> >
> > Thanks for the great work here.
> >
> > Confirming the v0.0.114 release number for the LDK "fake" lightning
> channels mitigations.
> >
> >  From the LDK-side, the vulnerability didn't come as novel knowledge, we
> have thought of potential
> > DoS issues in peer state machine handling (e.g [0]) a long time ago and
> our long-term design
> > philosophy has always been to privilege watchtower and process
> separation as a defense in-depth
> > mitigation (e.g see our glossary about "monitor replica" [1]). All those
> hardening architectures are
> > not implemented yet as part of the "vanilla" LDK (we're a library
> after), though it's consistent
> > with the answer we made privately during your disclosure.
> >
> > About the lessons, a few remarks.
> >
> > "Use watchtowers": note there is difference between watchtower only
> encompassing revoked state
> > punishment and "monitor replica' encompassing second-stage HTLC. For
> that type of DoS issues, you
> > wish to have the second deployed.
> >
> > "Multiple process": note your Lightning node multi-process should be
> free of "deadlock" and other
> > processing contaminating bugs, i.e the chain monitoring and reaction
> logic should maintain
> > liveliness even if the off-chain state machine coordinator (let's say
> `ChannelManager`) got DoS /
> > crashed, the chain monitoring should be able to detect revoked states
> and react finally.
> >
> > "More security auditing needed": I can only share the opinion that
> security and robustness has not
> > been the top priorities of the lightning implementations, for a very
> long time, I think
> > implementations have been more focus on spec features parity to maintain
> a usage market share,
> > rather thinking about the long-term network sustainability and safety of
> end-user funds. For the
> > more senior Lightning devs, those issues won’t come as strong surprise,
> I think some things like
> > package relay rank higher on folks priorities to mitigate publicly known
> and more serious security
> > issues [2].
> >
> > Looking forward to more security auditing of the Lightning Network.
> >
> > Cheers,
> > Antoine
> >
> > [0] https://github.com/lightningdevkit/rust-lightning/issues/383
> >  and
> > https://github.com/lightningdevkit/rust-lightning/issues/59
> > 
> > [1]
> https://github.com/lightningdevkit/rust-lightning/blob/main/GLOSSARY.md#monitor-replicas
> > <
> https://github.com/lightningdevkit/rust-lightning/blob/main/GLOSSARY.md#monitor-replicas
> >
> > [2] https://github.com/bitcoin/bips/pull/1382 <
> https://github.com/bitcoin/bips/pull/1382>
> >
> > Le jeu. 24 août 2023 à 01:36, Matt Morehouse  > > a écrit :
> >
> > Hi list,
> >
> > Last year a DoS vector was discovered to affect all node
> > implementations, where an attacker could open large numbers of fake
> > channels to the victim and never publish them on-chain, causing the
> > victim to consume excessive resources.
> >
> > Defenses against the DoS have been shipped in the following releases:
> > - CLN 23.02 [1]
> > - eclair 0.9.0 [2]
> > - LDK 0.0.114 [3]
> > - LND 0.16.0 [4]
> >
> > If you are running node software older than this, your funds may be
> at
> > risk!  Update to at least the above versions to help protect your
> > node.
> >
> > More details about the DoS vector are available at
> > https://morehouse.github.io/lightning/fake-channel-dos
> > .
> >
> > - Matt
> >
> > [1] https://github.com/ElementsProject/lightning/releases/tag/v23.02
> > 
> > [2] https://github.com/ACINQ/eclair/releases/tag/v0.9.0
> > 

Re: [Lightning-dev] Disclosure: Fake channel DoS vector

[Matt Corallo]

While you were aware of these fixes at the time, I'd appreciate it if you, someone who hasn't spent 
much time contributing to LDK over the past two or three years, stop trying to speak on behalf of 
the LDK project.


Thanks,
Matt

On 8/24/23 4:33 PM, Antoine Riard wrote:

Hi Matt,

Thanks for the great work here.

Confirming the v0.0.114 release number for the LDK "fake" lightning channels 
mitigations.

 From the LDK-side, the vulnerability didn't come as novel knowledge, we have thought of potential 
DoS issues in peer state machine handling (e.g [0]) a long time ago and our long-term design 
philosophy has always been to privilege watchtower and process separation as a defense in-depth 
mitigation (e.g see our glossary about "monitor replica" [1]). All those hardening architectures are 
not implemented yet as part of the "vanilla" LDK (we're a library after), though it's consistent 
with the answer we made privately during your disclosure.


About the lessons, a few remarks.

"Use watchtowers": note there is difference between watchtower only encompassing revoked state 
punishment and "monitor replica' encompassing second-stage HTLC. For that type of DoS issues, you 
wish to have the second deployed.


"Multiple process": note your Lightning node multi-process should be free of "deadlock" and other 
processing contaminating bugs, i.e the chain monitoring and reaction logic should maintain 
liveliness even if the off-chain state machine coordinator (let's say `ChannelManager`) got DoS / 
crashed, the chain monitoring should be able to detect revoked states and react finally.


"More security auditing needed": I can only share the opinion that security and robustness has not 
been the top priorities of the lightning implementations, for a very long time, I think 
implementations have been more focus on spec features parity to maintain a usage market share, 
rather thinking about the long-term network sustainability and safety of end-user funds. For the 
more senior Lightning devs, those issues won’t come as strong surprise, I think some things like 
package relay rank higher on folks priorities to mitigate publicly known and more serious security 
issues [2].


Looking forward to more security auditing of the Lightning Network.

Cheers,
Antoine

[0] https://github.com/lightningdevkit/rust-lightning/issues/383 
 and 
https://github.com/lightningdevkit/rust-lightning/issues/59 

[1] https://github.com/lightningdevkit/rust-lightning/blob/main/GLOSSARY.md#monitor-replicas 


[2] https://github.com/bitcoin/bips/pull/1382 


Le jeu. 24 août 2023 à 01:36, Matt Morehouse > a écrit :


Hi list,

Last year a DoS vector was discovered to affect all node
implementations, where an attacker could open large numbers of fake
channels to the victim and never publish them on-chain, causing the
victim to consume excessive resources.

Defenses against the DoS have been shipped in the following releases:
- CLN 23.02 [1]
- eclair 0.9.0 [2]
- LDK 0.0.114 [3]
- LND 0.16.0 [4]

If you are running node software older than this, your funds may be at
risk!  Update to at least the above versions to help protect your
node.

More details about the DoS vector are available at
https://morehouse.github.io/lightning/fake-channel-dos
.

- Matt

[1] https://github.com/ElementsProject/lightning/releases/tag/v23.02

[2] https://github.com/ACINQ/eclair/releases/tag/v0.9.0

[3] https://github.com/lightningdevkit/rust-lightning/releases/tag/v0.0.114

[4] https://github.com/lightningnetwork/lnd/releases/tag/v0.16.0-beta

___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org 

https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev



___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] Disclosure: Fake channel DoS vector

[Antoine Riard]

Hi Matt,

Thanks for the great work here.

Confirming the v0.0.114 release number for the LDK "fake" lightning
channels mitigations.

>From the LDK-side, the vulnerability didn't come as novel knowledge, we
have thought of potential DoS issues in peer state machine handling (e.g
[0]) a long time ago and our long-term design philosophy has always been to
privilege watchtower and process separation as a defense in-depth
mitigation (e.g see our glossary about "monitor replica" [1]). All those
hardening architectures are not implemented yet as part of the "vanilla"
LDK (we're a library after), though it's consistent with the answer we made
privately during your disclosure.

About the lessons, a few remarks.

"Use watchtowers": note there is difference between watchtower only
encompassing revoked state punishment and "monitor replica' encompassing
second-stage HTLC. For that type of DoS issues, you wish to have the second
deployed.

"Multiple process": note your Lightning node multi-process should be free
of "deadlock" and other processing contaminating bugs, i.e the chain
monitoring and reaction logic should maintain liveliness even if the
off-chain state machine coordinator (let's say `ChannelManager`) got DoS /
crashed, the chain monitoring should be able to detect revoked states and
react finally.

"More security auditing needed": I can only share the opinion that security
and robustness has not been the top priorities of the lightning
implementations, for a very long time, I think implementations have been
more focus on spec features parity to maintain a usage market share, rather
thinking about the long-term network sustainability and safety of end-user
funds. For the more senior Lightning devs, those issues won’t come as
strong surprise, I think some things like package relay rank higher on
folks priorities to mitigate publicly known and more serious security
issues [2].

Looking forward to more security auditing of the Lightning Network.

Cheers,
Antoine

[0] https://github.com/lightningdevkit/rust-lightning/issues/383 and
https://github.com/lightningdevkit/rust-lightning/issues/59
[1]
https://github.com/lightningdevkit/rust-lightning/blob/main/GLOSSARY.md#monitor-replicas
[2] https://github.com/bitcoin/bips/pull/1382

Le jeu. 24 août 2023 à 01:36, Matt Morehouse  a
écrit :

> Hi list,
>
> Last year a DoS vector was discovered to affect all node
> implementations, where an attacker could open large numbers of fake
> channels to the victim and never publish them on-chain, causing the
> victim to consume excessive resources.
>
> Defenses against the DoS have been shipped in the following releases:
> - CLN 23.02 [1]
> - eclair 0.9.0 [2]
> - LDK 0.0.114 [3]
> - LND 0.16.0 [4]
>
> If you are running node software older than this, your funds may be at
> risk!  Update to at least the above versions to help protect your
> node.
>
> More details about the DoS vector are available at
> https://morehouse.github.io/lightning/fake-channel-dos.
>
> - Matt
>
> [1] https://github.com/ElementsProject/lightning/releases/tag/v23.02
> [2] https://github.com/ACINQ/eclair/releases/tag/v0.9.0
> [3]
> https://github.com/lightningdevkit/rust-lightning/releases/tag/v0.0.114
> [4] https://github.com/lightningnetwork/lnd/releases/tag/v0.16.0-beta
> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev