Re: [Lightning-dev] [bitcoin-dev] A suggestion to periodically destroy (or remove to secondary storage for Archiving reasons) dust, Non-standard UTXOs, and also detected burn
Good Afternoon,
I am briefly reading the suggestion subject titled this email message.
The problem this idea addresses is not new, it is as old as computer
science with complexity and security varying from unused products in a
supermarket database to unused bank accounts and records of transactions
for archival. In the former, it is of no consequence to remove unused
products from the database if the itemised sales history is not
necessary and the report data is maintained ie. where the reporting is
stored generated. Once the products are removed it is impossible to
regenerate the detailed reports that called on the product-specific and
sales information. Archival works in a manner to remove data from
commonly used tables and to optimise them and if the data is later
required it can be recalled from larger possibly slower storage, or
simply kept in a larger less optimised table, in either case, all of the
data is still available. In the latter, it is a matter of consequence
and although archival is possible it is still necessary to ensure that
all archives are backed up, and the data can never be removed. This is
because if in an example transaction data is deleted after seven years
then it is no longer possible to see how an account has its balance and
an empty account with no transactions may be removed while actually
still holding a significant balance. If a mistake is made the result is
the same. This is because we allowed deletion of accounting records.
Actually, the recommendation from Computer Science all along is the same
as our case with Bitcoin - nothing should be deleted from the Blockchain
forever. For one substantiative reason, this is because it is necessary
for any client to be able to validate the blockchain in its entirety and
some clients may only be receiving information as to the state of the
blockchain from you. When you keep the entire blockchain you validate to
everybody else that you have the correct records. I arbitrarily object
to the use of pruned nodes but find them useful since the process of
validation is completed in its entirety when a new node comes online
even if it is pruned, but if only pruned nodes are available then
everybody has to believe the other nodes and this is unacceptable for
Bitcoin. It should be if some node is archiving UTXO's then it should
count as a pruned node, but my suggestion is, instead of making a
programming change just set your node with the parameter `prune=1` if
you wish to allow manually pruning from the Blockchain to a specific
height when you wish, or `prune= {>550} automatically prune blocks to
stay under target size in MiB`. My chain state database after all is
only 4.9GB and is hardly a concern for any operation of the standard
Bitcoin Core client. The answer, again, is you should just leave it and
get used to dealing with the bigger database.
KING JAMES HRMH
Great British Empire
Regards,
The Australian
LORD HIS EXCELLENCY JAMES HRMH (& HMRH)
of Hougun Manor & Glencoe & British Empire
MR. Damian A. James Williamson
Wills
et al.
Willtech
www.willtech.com.au
www.go-overt.com
duigco.org DUIGCO API
and other projects
m. 0487135719
f. +61261470192
This email does not constitute a general advice. Please disregard this
email if misdelivered.
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Lightning and other layer 2 projects with multiple RBF policies
> I suspect as with defaults generally most users will run whatever the > defaults are as they won't care to change them (or even be capable of > changing them if they are very non-technical). 30% nodes are using 0.21.1 right now whereas latest version was 22.0 and some are even running lower versions. Different versions in future with defaults might be running RBF v1 and RBF v2. > But users who have a stake in the security of Lightning (or other Layer 2 > projects) will clearly want to run whatever policy rules are beneficial to > those protocols. Agree and attackers will want to run the nodes with policy that helps them exploit bitcoin projects. Miners can run nodes with policy that helps them get more fees. > As you know the vast majority of the full nodes on the network currently run > Bitcoin Core. Whether that will change in future and whether this a good > thing or not is a whole other discussion. But the reality is that with such > strong dominance there is the option to set defaults that are widely used. Bitcoin Core with different versions are used at any point and not sure if this will ever change. https://luke.dashjr.org/programs/bitcoin/files/charts/security.html https://www.shodan.io/search/facet.png?query=User-Agent%3A%2FSatoshi%2F+port%3A%228333%22=product > I think if certain defaults can bolster the security of Lightning (and > possibly other Layer 2 projects) at no cost to full node users with no > interest in those protocols we should discuss what those defaults should be. This is the assumption which I don't agree with and hence asked some questions in my email. A new RBF policy used by default in Core will not improve the security of projects that are vulnerable to multiple RBF policies or rely on these policies in a way that affects their security. Maybe some experiments on signet might help in knowing more issues associated with multiple RBF policies. -- Prayank A3B1 E430 2298 178F Feb 13, 2022, 21:16 by michaelfolk...@protonmail.com: > Hi Prayank > > > 1.Is Lightning Network and a few other layer 2 projects vulnerable to > > multiple RBF policies being used? > > Clearly the security of the Lightning Network and some other Layer 2 projects > are at least impacted or partly dependent on policy rules in a way that the > base blockchain/network isn't. As I (and others) have said on many occasions > ideally this wouldn't be the case but it is best we can do with current > designs. I (and others) take the view that this is not a reason to abandon > those designs in the absence of an alternative that offers a strictly > superior security model. Going back to a model where *all* activity is > onchain (or even in less trust minimized protocols than Lightning) doesn't > seem like the right approach to me. > > > 2.With recent discussion to change things in default RBF policy used by > > Core, will we have multiple versions using different policies? Are users > > and especially miners incentivized to use different versions and policies? > > Do they have freedom to use different RBF policy? > > Without making policy rules effective consensus rules users (including > miners) are free to run different policy rules. I think it is too early to > say what the final incentives will be to run the same or differing policies. > Research into Lightning security is still nascent and we have no idea whether > alternative Layer 2 projects will thrive and whether they will have the same > or conflicting security considerations to Lightning. > > As you know the vast majority of the full nodes on the network currently run > Bitcoin Core. Whether that will change in future and whether this a good > thing or not is a whole other discussion. But the reality is that with such > strong dominance there is the option to set defaults that are widely used. I > think if certain defaults can bolster the security of Lightning (and possibly > other Layer 2 projects) at no cost to full node users with no interest in > those protocols we should discuss what those defaults should be. > > > 3.Are the recent improvements suggested for RBF policy only focused on > > Lightning Network and its security which will anyway remain same or become > > worse with multiple RBF policies? > > I think by nature of the Lightning Network being the most widely adopted > Layer 2 project most of the focus has been on Lightning security. But > contributors to other Layer 2 projects are free to flag and discuss security > considerations that aren't Lightning specific. > > > Note: Bitcoin Knots policy is fully configurable, even in the GUI - users > > can readily choose whatever policy *they* want. > > The maintainer(s) and contributors to Bitcoin Knots are free to determine > what default policy rules they want to implement (and make it easier for > users to change those defaults) in the absence of those policy rules being > made effective consensus rules. I suspect there
Re: [Lightning-dev] [bitcoin-dev] Thoughts on fee bumping, [bitcoin-dev] [Pre-BIP] Fee Accounts
Good Afternoon, I wish to post this discussion back to both threads to save repeating. As Peter Todd pointed out there are already two ways to increase the fee on a transaction RBF and Child Pays for Parent. Both of these methods are secure and do not allow for attack. Someone said there is not attack, however, sponsoring transactions allows for anyone to arbitrarily attach fees to a transaction in mempool as there is now way to validate that two UTXO's are associated as there are in the two cases already implemented, and this vector allows exploit if unchecked. A miner could arbitrarily attach 1BTC to a transaction is his own mempool, in fact if the miner has sufficient bitcoin all transactions he intends to mine, or in fact all miners could, and none of them need to broadcast the supplementary fee as mempool is gossip. Then when the miner is successful in creating a block all sponsored fees are returned and it is necessary for legitimate transactions to include a large fee in order to be selected, which may take to form sponsorship. This miners-only attack allows fees to be arbitrarily driven up. As it is I guess fees are averaging 0.238 BTC per block with 1.7K transactions per block and a fee of 0.00014000 BTC per transaction and without block reward and this is sufficient to drive down power usage which needs to go down a lot more to be sustainable in our global environment. KING JAMES HRMH Great British Empire Regards, The Australian LORD HIS EXCELLENCY JAMES HRMH (& HMRH) of Hougun Manor & Glencoe & British Empire MR. Damian A. James Williamson Wills et al. Willtech www.willtech.com.au www.go-overt.com duigco.org DUIGCO API and other projects m. 0487135719 f. +61261470192 This email does not constitute a general advice. Please disregard this email if misdelivered. On 2022-02-10 21:26, Antoine Riard via bitcoin-dev wrote: Hi James, I fully agree on the need to reframe the conversation around mempools/fee-bumping/L2s though please see my following comments, it's far from simple! Layering on special cases, more carve-outs, and X and Y percentage thresholds is going to make reasoning about the mempool harder than it already is. I think that's true with a lot of (all ?) pieces of software, there is a trend towards complexification. As new Bitcoin use-cases such as LN or vaults appear, it's not surprising to see the base layer upper interfaces changing to support the requirements. Same with kernels, at beginning, you can have a basic memory support with paging/memory rights/kernel allocators then as you start to support more platforms/devices you might have to support swaps/DMA/VMs management... That we should keep the complexity reasonably sane to enable human auditing, and maybe learn from the failures of systems engineering, that's something to muse on. The countervailing force here ends up being spam prevention (a la min-relay-fee) to prevent someone from consuming bandwidth and mempool space with a long series of infinitesimal fee-bumps. I think here we should dissociate a) long-chain of transactions and b) high-number of repeated fee-bumps. For a) _if_ SIGHASH_ANYPREVOUT is deployed and Eltoo adopted as a primary update mechanism for stateful L2s, one might envision long-chain of update transactions servicing as a new pinning vector, where all the chain elements are offering a compelling feerate/fees. It might be solvable with smarter mempool logic sorting the elements from the best offer to the lower ones, though that issue would need more serious investigation. For b) if we bound with a hard constant the number of RBF attempts, we decrease the fault-tolerance of L2 transaction issuers. Some of them might connect directly to the miners because they're offering higher number of incentive-compatible RBF attempts than vanilla full-nodes. That might provoke a more or slow centralization of the transaction relay topology... Instead of prompting a rebroadcast of the original transaction for replacement, which contains a lot of data not new to the network, it makes more sense to broadcast the "diff" which is the additive contribution towards some txn's feerate. In a distributed system such as the Bitcoin p2p network, you might have transaction A and transaction B broadcast at the same time and your peer topology might fluctuate between original send and broadcast of the diff, you don't know who's seen what... You might inefficiently announce diff A on top of B and diff B on top A. We might leverage set reconciliation there a la Erlay, though likely with increased round-trips. It's probably uncontroversial at this point to say that even RBF itself is kind of a hack - a special sequence number should not be necessary for post-broadcast contribution toward feerate. I think here we should dissociate the replace-by-fee mechanism itself from the replacement signaling one. To have a functional RBF, you don't need signaling at all, just consider all
Re: [Lightning-dev] [bitcoin-dev] [Pre-BIP] Fee Accounts
Good Afternoon, Thank-you Jeremy for your work on this, it is valuable for the public consideration but unfortunately I disagree. The idea of being able to arbitrarily attach a fee to a transaction allows a miners-only attack in which all sponsored transactions are mined and the fee-rate can be manipulated. It is possible that it will also allow other exotic forms of attack. It is true that is seems like a drawback that we cannot see the future fee rate when we create a transaction but given the value of Bitcoin it seems more likely that we will have overpaid for the fee component and our transactions will be sort after to make the most valuable blocks. If somehow you disagree, I would be interested to hear how it would not make a problem? KING JAMES HRMH Great British Empire Regards, The Australian LORD HIS EXCELLENCY JAMES HRMH (& HMRH) of Hougun Manor & Glencoe & British Empire MR. Damian A. James Williamson Wills et al. Willtech www.willtech.com.au www.go-overt.com duigco.org DUIGCO API and other projects m. 0487135719 f. +61261470192 This email does not constitute a general advice. Please disregard this email if misdelivered. -- On 2022-02-10 17:58, Peter Todd via bitcoin-dev wrote: On Sat, Jan 01, 2022 at 12:04:00PM -0800, Jeremy via bitcoin-dev wrote: Happy new years devs, I figured I would share some thoughts for conceptual review that have been bouncing around my head as an opportunity to clean up the fee paying semantics in bitcoin "for good". The design space is very wide on the approach I'll share, so below is just a sketch of how it could work which I'm sure could be improved greatly. Transaction fees are an integral part of bitcoin. However, due to quirks of Bitcoin's transaction design, fees are a part of the transactions that they occur in. While this works in a "Bitcoin 1.0" world, where all transactions are simple on-chain transfers, real world use of Bitcoin requires support for things like Fee Bumping stuck transactions, DoS resistant Payment Channels, and other long lived Smart Contracts that can't predict future fee rates. Having the fees paid in band makes writing these contracts much more difficult as you can't merely express the logic you want for the transaction, but also the fees. Previously, I proposed a special type of transaction called a "Sponsor" which has some special consensus + mempool rules to allow arbitrarily appending fees to a transaction to bump it up in the mempool. As an alternative, we could establish an account system in Bitcoin as an "extension block". This type of design works really well for channels because the addition of fees to e.g. a channel state does not require any sort of pre-planning (e.g. anchors) or transaction flexibility (SIGHASH flags). This sort of design is naturally immune to pinning issues since you could offer to pay a fee for any TXID and the number of fee adding offers does not need to be restricted in the same way the descendant transactions would need to be. So it's important to recognize that fee accounts introduce their own kind of transaction pinning attacks: third parties would be able to attach arbitrary fees to any transaction without permission. This isn't necessarily a good thing: I don't want third parties to be able to grief my transaction engines by getting obsolete transactions confirmed in liu of the replacments I actually want confirmed. Eg a third party could mess up OpenTimestamps calendars at relatively low cost by delaying the mining of timestamp txs. Of course, there's an obvious way to fix this: allow transactions to designate a pubkey allowed to add further transaction fees if required. Which Bitcoin already has in two forms: Replace-by-Fee and Child Pays for Parent. ___ bitcoin-dev mailing list bitcoin-...@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Lightning and other layer 2 projects with multiple RBF policies
Hi Prayank > 1.Is Lightning Network and a few other layer 2 projects vulnerable to > multiple RBF policies being used? Clearly the security of the Lightning Network and some other Layer 2 projects are at least impacted or partly dependent on policy rules in a way that the base blockchain/network isn't. As I (and others) have said on many occasions ideally this wouldn't be the case but it is best we can do with current designs. I (and others) take the view that this is not a reason to abandon those designs in the absence of an alternative that offers a strictly superior security model. Going back to a model where *all* activity is onchain (or even in less trust minimized protocols than Lightning) doesn't seem like the right approach to me. > 2.With recent discussion to change things in default RBF policy used by Core, > will we have multiple versions using different policies? Are users and > especially miners incentivized to use different versions and policies? Do > they have freedom to use different RBF policy? Without making policy rules effective consensus rules users (including miners) are free to run different policy rules. I think it is too early to say what the final incentives will be to run the same or differing policies. Research into Lightning security is still nascent and we have no idea whether alternative Layer 2 projects will thrive and whether they will have the same or conflicting security considerations to Lightning. I suspect as with defaults generally most users will run whatever the defaults are as they won't care to change them (or even be capable of changing them if they are very non-technical). But users who have a stake in the security of Lightning (or other Layer 2 projects) will clearly want to run whatever policy rules are beneficial to those protocols. As you know the vast majority of the full nodes on the network currently run Bitcoin Core. Whether that will change in future and whether this a good thing or not is a whole other discussion. But the reality is that with such strong dominance there is the option to set defaults that are widely used. I think if certain defaults can bolster the security of Lightning (and possibly other Layer 2 projects) at no cost to full node users with no interest in those protocols we should discuss what those defaults should be. > 3.Are the recent improvements suggested for RBF policy only focused on > Lightning Network and its security which will anyway remain same or become > worse with multiple RBF policies? I think by nature of the Lightning Network being the most widely adopted Layer 2 project most of the focus has been on Lightning security. But contributors to other Layer 2 projects are free to flag and discuss security considerations that aren't Lightning specific. > Note: Bitcoin Knots policy is fully configurable, even in the GUI - users can > readily choose whatever policy *they* want. The maintainer(s) and contributors to Bitcoin Knots are free to determine what default policy rules they want to implement (and make it easier for users to change those defaults) in the absence of those policy rules being made effective consensus rules. I suspect there would be strong opposition to making some policy rules effective consensus rules but we are now venturing again into future speculation and none of us have a crystal ball. Certainly if you take the view that these policy rules should never be made effective consensus rules then the fact there is at least one implementation taking a contrasting approach to Core is a good thing. -- Michael Folkson Email: michaelfolkson at [protonmail.com](http://protonmail.com/) Keybase: michaelfolkson PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3 --- Original Message --- On Sunday, February 13th, 2022 at 6:09 AM, Prayank via Lightning-dev wrote: > Hello World, > > There was a discussion about improving fee estimation in Bitcoin Core last > year in which 'instagibbs' mentioned that we cannot consider mempool as an > orderbook in which which everyone is bidding for block space because nodes > can use different relay policies: > https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2021-09-22#706294; > > Although I still don't consider fee rates used in last few blocks relevant > for fee estimation, it is possible that we have nodes with different relay > policies. > > Similarly if we have different RBF policies being used by nodes in future, > how would this affect the security of lightning network implementations and > other layer 2 projects? > > Based on the things shared by 'aj' in > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-February/019846.html > it is possible for an attacker to use a different RBF policy with some > nodes, 10% hash power and affect the security of different projects that rely > on default RBF policy in latest Bitcoin Core. > > There was even a CVE in which RBF policy not being documented
