Re: [Lightning-dev] On the scalability issues of onboarding millions of LN mobile clients
Hi Antoine, > Even with cheaper, more efficient protocols like BIP 157, you may have a > huge discrepancy between what is asked and what is offered. Assuming 10M > light clients [0] each of them consuming ~100MB/month for filters/headers, > that means you're asking 1PB/month of traffic to the backbone network. If > you assume 10K public nodes, like today, assuming _all_ of them opt-in to > signal BIP 157, that's an increase of 100GB/month for each. Which is > consequent with regards to the estimated cost of 350GB/month for running > an actual public node One really dope thing about BIP 157+158, is that the protocol makes serving light clients now _stateless_, since the full node doesn't need to perform any unique work for a given client. As a result, the entire protocol could be served over something like HTTP, taking advantage of all the established CDNs and anycast serving infrastructure, which can reduce syncing time (less latency to fetch data) and also more widely distributed the load of light clients using the existing web infrastructure. Going further, with HTTP/2's server-push capabilities, those serving this data can still push out notifications for new headers, etc. > Therefore, you may want to introduce monetary compensation in exchange of > servicing filters. Light client not dedicating resources to maintain the > network but free-riding on it, you may use their micro-payment > capabilities to price chain access resources [3] Piggy backing off the above idea, if the data starts being widely served over HTTP, then LSATs[1][2] can be used to add a lightweight payment mechanism by inserting a new proxy server in front of the filter/header infrastructure. The minted tokens themselves may allow a user to purchase access to a single header/filter, a range of them in the past, or N headers past the known chain tip, etc, etc. -- Laolu [1]: https://lsat.tech/ [2]: https://lightning.engineering/posts/2020-03-30-lsat/ On Tue, May 5, 2020 at 3:17 AM Antoine Riard wrote: > Hi, > > (cross-posting as it's really both layers concerned) > > Ongoing advancement of BIP 157 implementation in Core maybe the > opportunity to reflect on the future of light client protocols and use this > knowledge to make better-informed decisions about what kind of > infrastructure is needed to support mobile clients at large scale. > > Trust-minimization of Bitcoin security model has always relied first and > above on running a full-node. This current paradigm may be shifted by LN > where fast, affordable, confidential, censorship-resistant payment services > may attract a lot of adoption without users running a full-node. Assuming a > user adoption path where a full-node is required to benefit for LN may > deprive a lot of users, especially those who are already denied a real > financial infrastructure access. It doesn't mean we shouldn't foster node > adoption when people are able to do so, and having a LN wallet maybe even a > first-step to it. > > Designing a mobile-first LN experience opens its own gap of challenges > especially in terms of security and privacy. The problem can be scoped as > how to build a scalable, secure, private chain access backend for millions > of LN clients ? > > Light client protocols for LN exist (either BIP157 or Electrum are used), > although their privacy and security guarantees with regards to > implementation on the client-side may still be an object of concern > (aggressive tx-rebroadcast, sybillable outbound peer selection, trusted fee > estimation). That said, one of the bottlenecks is likely the number of > full-nodes being willingly to dedicate resources to serve those clients. > It's not about _which_ protocol is deployed but more about _incentives_ for > node operators to dedicate long-term resources to client they have lower > reasons to care about otherwise. > > Even with cheaper, more efficient protocols like BIP 157, you may have a > huge discrepancy between what is asked and what is offered. Assuming 10M > light clients [0] each of them consuming ~100MB/month for filters/headers, > that means you're asking 1PB/month of traffic to the backbone network. If > you assume 10K public nodes, like today, assuming _all_ of them opt-in to > signal BIP 157, that's an increase of 100GB/month for each. Which is > consequent with regards to the estimated cost of 350GB/month for running an > actual public node. Widening full-node adoption, specially in term of > geographic distribution means as much as we can to bound its operational > cost. > > Obviously, deployment of more efficient tx-relay protocol like Erlay will > free up some resources but it maybe wiser to dedicate them to increase > health and security of the backbone network like deploying more outbound > connections. > > Unless your light client protocol is so ridiculous cheap to rely on > niceness of a subset of node operators offering free resources, it won't > scale. And it's likely you will always have a ratio disequilibri
Re: [Lightning-dev] Force close of channel with unresolved htlc
Dear Subhra, as discussed bilaterally and after clarification of your question the situation is as follows: Let us assume A and B have a channel in which A has 4 tokens and B has 6 tokens Now A offers an HTLC with the amount of 2 tokens and B accepts (receives) the offer then A and B both have negotiated the HTLC output in the most recent commitment transaction. If A stops responding and B has to force close the channel a commitment transaction with 3 UTXOs will hit the chain. One UTXO with 2 tokens spendable by A, another one with 6 tokens spendable by B and the received HTLC output with 2 tokens. This one can be spend by two different conditions as in the offchain protocol 1.) Before the timelock of the HTLC has passed B can spend the output if B knows his to_local HTLC secret AND the preimage. OR 2.) after the timelock A can spend the output if A knows the to_remote HTLC secret. the mechanism with HTLCs can be read upon in BOLT 2 (channel operation https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md) and the scripts can be seen in BOLT 3: https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md A less technical summary that is more focused on explaining the concepts is currently being developed in the routing chapter of mastering the lightning network: https://github.com/lnbook/lnbook/blob/43ce57298b4da345286ae3b53c42ea3eb9d9b056/routing.asciidoc With kind regards Rene Pickhardt On Tue, May 5, 2020 at 8:06 PM Subhra Mazumdar < subhra.mazumdar1...@gmail.com> wrote: > Hi, > I am having a doubt regarding force closure of channel. Suppose A->B > there is an htlc which has been established for transfering fund. Now > suppose for some unfortunate reason B doesnt have the witness to resolve > htlc and the mean time A suffers crash fault. Then can B close the channel > given that it has no way out of resolving the htlc due to lack of witness? > ___ > Lightning-dev mailing list > Lightning-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > -- https://www.rene-pickhardt.de Skype: rene.pickhardt mobile: +49 (0)176 5762 3618 ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
[Lightning-dev] Force close of channel with unresolved htlc
Hi, I am having a doubt regarding force closure of channel. Suppose A->B there is an htlc which has been established for transfering fund. Now suppose for some unfortunate reason B doesnt have the witness to resolve htlc and the mean time A suffers crash fault. Then can B close the channel given that it has no way out of resolving the htlc due to lack of witness? ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] On the scalability issues of onboarding millions of LN mobile clients
On Tue, May 5, 2020 at 9:01 PM Luke Dashjr via bitcoin-dev < bitcoin-...@lists.linuxfoundation.org> wrote: > On Tuesday 05 May 2020 10:17:37 Antoine Riard via bitcoin-dev wrote: > > Trust-minimization of Bitcoin security model has always relied first and > > above on running a full-node. This current paradigm may be shifted by LN > > where fast, affordable, confidential, censorship-resistant payment > services > > may attract a lot of adoption without users running a full-node. > > No, it cannot be shifted. This would compromise Bitcoin itself, which for > security depends on the assumption that a supermajority of the economy is > verifying their incoming transactions using their own full node. > Hi Luke, I have heard this claim made several times but have never understood the argument behind it. The question I always have is: If I get scammed by not verifying my incoming transactions properly how can this affect anyone else? It's very unintuative. I've been scammed several times in my life in fiat currency transactions but as far as I could tell it never negatively affected the currency overall! The links you point and from what I've seen you say before refer to "miner control" as the culprit. My only thought is that this is because a light client could follow a dishonest majority of hash power chain. But this just brings me back to the question. If, instead of BTC, I get a payment in some miner scamcoin on their dishonest fork (but I think it's BTC because I'm running a light client) that still seems to only to damage me. Where does the side effect onto others on the network come from? Cheers, LL ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] On the scalability issues of onboarding millions of LN mobile clients
Good morning ariard and luke-jr > > Trust-minimization of Bitcoin security model has always relied first and > > above on running a full-node. This current paradigm may be shifted by LN > > where fast, affordable, confidential, censorship-resistant payment services > > may attract a lot of adoption without users running a full-node. > > No, it cannot be shifted. This would compromise Bitcoin itself, which for > security depends on the assumption that a supermajority of the economy is > verifying their incoming transactions using their own full node. > > The past few years has seen severe regressions in this area, to the point > where Bitcoin's future seems quite bleak. Without serious improvements to the > full node ratio, Bitcoin is likely to fail. > > Therefore, all efforts to improve the "full node-less" experience are harmful, > and should be actively avoided. BIP 157 improves privacy of fn-less usage, > while providing no real benefits to full node users (compared to more > efficient protocols like Stratum/Electrum). > > For this reason, myself and a few others oppose merging support for BIP 157 in > Core. BIP 157 can be implemented as a separate daemon that processes the blocks downloaded by an attached `bitcoind`, i.e. what Wasabi does. The intention, as I understood it, of putting BIP157 directly into bitcoind was to essentially force all `bitcoind` users to possibly service BIP157 clients, in the hope that a BIP157 client can contact any arbitrary fullnode to get BIP157 service. This is supposed to improve to the situation relative to e.g. Electrum, where there are far fewer Electrum servers than fullnodes. Of course, as ariard computes, deploying BIP157 could lead to an effective DDoS on the fullnode network if a large number of BIP157 clients arise. Though maybe this will not occur very fast? We hope? It seems to me that the thing that *could* be done would be to have watchtowers provide light-client services, since that seems to be the major business model of watchtowers, as suggested by ariard as well. This is still less than ideal, but maybe is better than nothing. Regards, ZmnSCPxj ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] [bitcoin-dev] On the scalability issues of onboarding millions of LN mobile clients
On Tuesday 05 May 2020 10:17:37 Antoine Riard via bitcoin-dev wrote: > Trust-minimization of Bitcoin security model has always relied first and > above on running a full-node. This current paradigm may be shifted by LN > where fast, affordable, confidential, censorship-resistant payment services > may attract a lot of adoption without users running a full-node. No, it cannot be shifted. This would compromise Bitcoin itself, which for security depends on the assumption that a supermajority of the economy is verifying their incoming transactions using their own full node. The past few years has seen severe regressions in this area, to the point where Bitcoin's future seems quite bleak. Without serious improvements to the full node ratio, Bitcoin is likely to fail. Therefore, all efforts to improve the "full node-less" experience are harmful, and should be actively avoided. BIP 157 improves privacy of fn-less usage, while providing no real benefits to full node users (compared to more efficient protocols like Stratum/Electrum). For this reason, myself and a few others oppose merging support for BIP 157 in Core. > Assuming a user adoption path where a full-node is required to benefit for > LN may deprive a lot of users, especially those who are already denied a > real financial infrastructure access. If Bitcoin can't do it, then Bitcoin can't do it. Bitcoin can't solve *any* problem if it becomes insecure itself. Luke P.S. See also https://medium.com/@nicolasdorier/why-i-dont-celebrate-neutrino-206bafa5fda0 https://medium.com/@nicolasdorier/neutrino-is-dangerous-for-my-self-sovereignty-18fac5bcdc25 ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] On the scalability issues of onboarding millions of LN mobile clients
Hey Antoine, just a small note, [3] is missing in your footnotes, can you add it? Thanks On Tue, 5 May 2020 at 18:17, Antoine Riard wrote: > Hi, > > (cross-posting as it's really both layers concerned) > > Ongoing advancement of BIP 157 implementation in Core maybe the > opportunity to reflect on the future of light client protocols and use this > knowledge to make better-informed decisions about what kind of > infrastructure is needed to support mobile clients at large scale. > > Trust-minimization of Bitcoin security model has always relied first and > above on running a full-node. This current paradigm may be shifted by LN > where fast, affordable, confidential, censorship-resistant payment services > may attract a lot of adoption without users running a full-node. Assuming a > user adoption path where a full-node is required to benefit for LN may > deprive a lot of users, especially those who are already denied a real > financial infrastructure access. It doesn't mean we shouldn't foster node > adoption when people are able to do so, and having a LN wallet maybe even a > first-step to it. > > Designing a mobile-first LN experience opens its own gap of challenges > especially in terms of security and privacy. The problem can be scoped as > how to build a scalable, secure, private chain access backend for millions > of LN clients ? > > Light client protocols for LN exist (either BIP157 or Electrum are used), > although their privacy and security guarantees with regards to > implementation on the client-side may still be an object of concern > (aggressive tx-rebroadcast, sybillable outbound peer selection, trusted fee > estimation). That said, one of the bottlenecks is likely the number of > full-nodes being willingly to dedicate resources to serve those clients. > It's not about _which_ protocol is deployed but more about _incentives_ for > node operators to dedicate long-term resources to client they have lower > reasons to care about otherwise. > > Even with cheaper, more efficient protocols like BIP 157, you may have a > huge discrepancy between what is asked and what is offered. Assuming 10M > light clients [0] each of them consuming ~100MB/month for filters/headers, > that means you're asking 1PB/month of traffic to the backbone network. If > you assume 10K public nodes, like today, assuming _all_ of them opt-in to > signal BIP 157, that's an increase of 100GB/month for each. Which is > consequent with regards to the estimated cost of 350GB/month for running an > actual public node. Widening full-node adoption, specially in term of > geographic distribution means as much as we can to bound its operational > cost. > > Obviously, deployment of more efficient tx-relay protocol like Erlay will > free up some resources but it maybe wiser to dedicate them to increase > health and security of the backbone network like deploying more outbound > connections. > > Unless your light client protocol is so ridiculous cheap to rely on > niceness of a subset of node operators offering free resources, it won't > scale. And it's likely you will always have a ratio disequilibrium between > numbers of clients and numbers of full-node, even worst their growth rate > won't be the same, first ones are so much easier to setup. > > It doesn't mean servicing filters for free won't work for now, numbers of > BIP157 clients is still pretty low, but what is worrying is wallet vendors > building such chain access backend, hitting a bandwidth scalability wall > few years from now instead of pursuing better solutions. And if this > happen, maybe suddenly, isn't the quick fix going to be to rely on > centralized services, so much easier to deploy ? > > Of course, it may be brought that actually current full-node operators > don't get anything back from servicing blocks, transactions, addresses... > It may be replied that you have an indirect incentive to participate in > network relay and therefore guarantee censorship-resistance, instead of > directly connecting to miners. You do have today ways to select your > resources exposure like pruning, block-only or being private but the wider > point is the current (non?)-incentives model seems to work for the base > layer. For light clients data, are node operators going to be satisfied to > serve this new *class* of traffic en masse ? > > This doesn't mean you won't find BIP157 servers, ready to serve you with > unlimited credit, but it's more likely their intentions maybe not aligned, > like spying on your transaction broadcast or block fetched. And you do want > peer diversity to avoid every BIP157 servers being on few ASNs for > fault-tolerance. Do people expect a scenario a la Cloudflare, where > everyone connections is to far or less the same set of entities ? > > Moreover, the LN security model diverges hugely from basic on-chain > transactions. Worst-case attack on-chain a malicious light client server > showing a longest, invalid, PoW-signed chain to double-spend the user.
[Lightning-dev] On the scalability issues of onboarding millions of LN mobile clients
Hi, (cross-posting as it's really both layers concerned) Ongoing advancement of BIP 157 implementation in Core maybe the opportunity to reflect on the future of light client protocols and use this knowledge to make better-informed decisions about what kind of infrastructure is needed to support mobile clients at large scale. Trust-minimization of Bitcoin security model has always relied first and above on running a full-node. This current paradigm may be shifted by LN where fast, affordable, confidential, censorship-resistant payment services may attract a lot of adoption without users running a full-node. Assuming a user adoption path where a full-node is required to benefit for LN may deprive a lot of users, especially those who are already denied a real financial infrastructure access. It doesn't mean we shouldn't foster node adoption when people are able to do so, and having a LN wallet maybe even a first-step to it. Designing a mobile-first LN experience opens its own gap of challenges especially in terms of security and privacy. The problem can be scoped as how to build a scalable, secure, private chain access backend for millions of LN clients ? Light client protocols for LN exist (either BIP157 or Electrum are used), although their privacy and security guarantees with regards to implementation on the client-side may still be an object of concern (aggressive tx-rebroadcast, sybillable outbound peer selection, trusted fee estimation). That said, one of the bottlenecks is likely the number of full-nodes being willingly to dedicate resources to serve those clients. It's not about _which_ protocol is deployed but more about _incentives_ for node operators to dedicate long-term resources to client they have lower reasons to care about otherwise. Even with cheaper, more efficient protocols like BIP 157, you may have a huge discrepancy between what is asked and what is offered. Assuming 10M light clients [0] each of them consuming ~100MB/month for filters/headers, that means you're asking 1PB/month of traffic to the backbone network. If you assume 10K public nodes, like today, assuming _all_ of them opt-in to signal BIP 157, that's an increase of 100GB/month for each. Which is consequent with regards to the estimated cost of 350GB/month for running an actual public node. Widening full-node adoption, specially in term of geographic distribution means as much as we can to bound its operational cost. Obviously, deployment of more efficient tx-relay protocol like Erlay will free up some resources but it maybe wiser to dedicate them to increase health and security of the backbone network like deploying more outbound connections. Unless your light client protocol is so ridiculous cheap to rely on niceness of a subset of node operators offering free resources, it won't scale. And it's likely you will always have a ratio disequilibrium between numbers of clients and numbers of full-node, even worst their growth rate won't be the same, first ones are so much easier to setup. It doesn't mean servicing filters for free won't work for now, numbers of BIP157 clients is still pretty low, but what is worrying is wallet vendors building such chain access backend, hitting a bandwidth scalability wall few years from now instead of pursuing better solutions. And if this happen, maybe suddenly, isn't the quick fix going to be to rely on centralized services, so much easier to deploy ? Of course, it may be brought that actually current full-node operators don't get anything back from servicing blocks, transactions, addresses... It may be replied that you have an indirect incentive to participate in network relay and therefore guarantee censorship-resistance, instead of directly connecting to miners. You do have today ways to select your resources exposure like pruning, block-only or being private but the wider point is the current (non?)-incentives model seems to work for the base layer. For light clients data, are node operators going to be satisfied to serve this new *class* of traffic en masse ? This doesn't mean you won't find BIP157 servers, ready to serve you with unlimited credit, but it's more likely their intentions maybe not aligned, like spying on your transaction broadcast or block fetched. And you do want peer diversity to avoid every BIP157 servers being on few ASNs for fault-tolerance. Do people expect a scenario a la Cloudflare, where everyone connections is to far or less the same set of entities ? Moreover, the LN security model diverges hugely from basic on-chain transactions. Worst-case attack on-chain a malicious light client server showing a longest, invalid, PoW-signed chain to double-spend the user. On LN, the *liveliness* requirement means the entity owning your view of the chain can lie to you on whether your channel has been spent by a revoked commitment, the real tip of the blockchain or even dry-up block announcement to trigger unexpected behavior in the client logic. A malicious ligh
