Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
You very clearly didn't bother to read other mails in this thread. To make it easy for you, here's a few links: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-July/017147.html https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-July/017175.html Matt > On Aug 13, 2019, at 23:05, Will Madden wrote: > > For the record, strong NACK. My understanding is that this breaks several > established SPV implementations (such as early breadwallet for sure and > possibly current BRD wallets) and I have yet to see quantitative > prioritization or even a rational justification for this change. > > Requiring SPV wallets to communicate with trusted nodes is centralization, > and breaking functionality and implementations that enable this without a > thoroughly researched rationale is highly suspect. > >> On Jul 20, 2019, at 1:46 PM, Matt Corallo via bitcoin-dev >> wrote: >> >> Just a quick heads-up for those watching the list who may be using it - >> in the next Bitcoin Core release bloom filter serving will be turned off >> by default. This has been a long time coming, it's been an option for >> many releases and has been a well-known DoS vector for some time. >> As other DoS vectors have slowly been closed, this has become >> increasingly an obvious low-hanging fruit. Those who are using it should >> already have long been filtering for NODE_BLOOM-signaling nodes, and I >> don't anticipate those being gone any time particularly soon. >> >> See-also PR at https://github.com/bitcoin/bitcoin/pull/16152 >> >> The release notes will liekly read: >> >> P2P Changes >> --- >> - The default value for the -peerbloomfilters configuration option (and, >> thus, NODE_BLOOM support) has been changed to false. >> This resolves well-known DoS vectors in Bitcoin Core, especially for >> nodes with spinning disks. It is not anticipated that >> this will result in a significant lack of availability of >> NODE_BLOOM-enabled nodes in the coming years, however, clients >> which rely on the availability of NODE_BLOOM-supporting nodes on the >> P2P network should consider the process of migrating >> to a more modern (and less trustful and privacy-violating) alternative >> over the coming years. >> >> Matt >> ___ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On Tuesday 23 July 2019 14:47:18 Andreas Schildbach via bitcoin-dev wrote: > 3) Afaik, it enforces/encourages address re-use. This stems from the > fact that the server decides on the filter and in particular on the > false positive rate. On wallets with many addresses, a hardcoded filter > will be too blurry and thus each block will be matched. So wallets that > follow the "one address per incoming payment" pattern (e.g. HD wallets) > at some point will be forced to wrap their key chains back to the > beginning. If I'm wrong on this one please let me know. BTW, you are indeed wrong on this. You don't need to match every single address the wallet has ever used, only outstanding addresses that haven't been paid. ;) Luke ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
This conversation went off the rails somewhat. I don't think there's any immediate risk of NODE_BLOOM peers being unavailable. This is a defaults change, not a removal of the code to serve BIP 37 peers (nor would I suggest removing said code while people still want to use them - the maintenance burden isn't much). Looking at historical upgrade cycles, ignoring any other factors, there will be a large number of nodes serving NODE_BLOOM for many years. Even more importantly, if you need them, run a node or two. As long as no one is exploiting the issues with them such a node isn't *too* expensive. Or don't, I guarantee you chainanalysis or some competitor of theirs will very very happily serve bloom-filtered clients as long as such clients want to deanonymize themselves. We already see a plurality of nodes on the network are clearly not run-of-the-mill Core nodes, many of which are likely deanonimization efforts. In some cases BIP 137 is a replacement, in some cases, indeed, it is not. I agree at a protocol level we shouldn't be passing judgement about how users wish to interact with the Bitcoin system (aside from not putting our own, personal, effort into building such things) but that isn't what's happening here. This is an important DoS fix for the average node, and I don't really understand the argument that this is going to break existing BIP 37 wallets, but if it makes you feel any better I can run some beefy BIP 37 nodes. Matt > On Jul 26, 2019, at 06:04, Jonas Schnelli via bitcoin-dev > wrote: > > >> 1) It causes way too much traffic for mobile users, and likely even too >> much traffic for fixed lines in not so developed parts of the world. > > Yes. It causes more traffic than BIP37. > Basic block filters for current last ~7 days (1008 blocks) are about 19MB > (just the filters). > On top, you will probably fetch a handful of irrelevant blocks due to the FPs > and due to true relevant txns. > A over-the-thumb estimation: ~25MB per week of catch-up. > If you where offline for a month: ~108MB > > Thats certainly more then BIP37 BF (measured 1.6MB total traffic with android > schildbach wallet restore blockchain for 8 week [7 weeks headers, 1week > merkleblocks]). > > But lets look at it like this: for an additional, say 25MB per week (maybe a > bit more), you get the ability to filter blocks without depending on serving > peers who may compromise your financial privacy. > Also, if you keep the filters, further rescans do consume the same or less > bandwidth than BF BIP37. > In other words: you have the chance to potentially increase privacy by > consuming bandwidth in the range of a single audio podcast per week. > > I would say the job of protocol developers is protect users privacy where > it’s possible (as a default). > It’s probably a debatable point wether 25MB per week of traffic is worth a > potential increase in privacy, though I absolutely think 25MB/week is an > acceptable tradeoff. > Saving traffic is possible by using BIP37 or stratum/electrum… but developers > should make sure users are __warned about the consequences__! > > Additionally, it looks like, peer operators are not endless being willing to > serve – for free – a CPU/disk intense service with no benefits for the > network. I would question wether a decentralised form of BIP37 is sustainable > in the long run (if SPV wallet provider bootstrap a net range of NODE_BLOOM > peers to make it more reliable on the network would be snake-oil). > > >> >> 2) It filters blocks only. It doesn't address unconfirmed transactions. > > Well, unconfirmed transaction are uncertain for various reasons. > > BIP158 won't allow you to filter the mempool. > But as soon as you are connected to the network, you may fetch tx with > inv/getdata and pick out the relevant ones (causes also traffic). > Unclear and probably impossible with the current BIP158 specs to fetch > transactions that are not in active relay and are not in a block (mempool > txns, at least this is true with the current observed relay tactics). > > >> 3) Afaik, it enforces/encourages address re-use. This stems from the >> fact that the server decides on the filter and in particular on the >> false positive rate. On wallets with many addresses, a hardcoded filter >> will be too blurry and thus each block will be matched. So wallets that >> follow the "one address per incoming payment" pattern (e.g. HD wallets) >> at some point will be forced to wrap their key chains back to the >> beginning. If I'm wrong on this one please let me know. > > I’m probably the wrong guy to ask (haven’t made the numbers) but last time I > rescanned a Core wallet (in my dev branch) with block filters (and a Core > wallet has >2000 addresses by default) it fetched a low and acceptable amount > of false positive blocks. > (Maybe someone who made the numbers step in here.) > > Though, large wallets – AFAIK – also operate badly with BIP37. > >> >> 4)
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On 7/23/19 10:47 AM, Andreas Schildbach via bitcoin-dev wrote: 3) Afaik, it enforces/encourages address re-use. This stems from the fact that the server decides on the filter and in particular on the false positive rate. On wallets with many addresses, a hardcoded filter will be too blurry and thus each block will be matched. So wallets that follow the "one address per incoming payment" pattern (e.g. HD wallets) at some point will be forced to wrap their key chains back to the beginning. If I'm wrong on this one please let me know. Maybe someone who knows better can confirm but I thought I read the the fp rate on the filter was chosen assuming a wallet would average a certain number of addresses (ie, they assumed use of an HD wallet). Your criticism might be valid for wallets that well exceed the average. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
> 1) It causes way too much traffic for mobile users, and likely even too > much traffic for fixed lines in not so developed parts of the world. Yes. It causes more traffic than BIP37. Basic block filters for current last ~7 days (1008 blocks) are about 19MB (just the filters). On top, you will probably fetch a handful of irrelevant blocks due to the FPs and due to true relevant txns. A over-the-thumb estimation: ~25MB per week of catch-up. If you where offline for a month: ~108MB Thats certainly more then BIP37 BF (measured 1.6MB total traffic with android schildbach wallet restore blockchain for 8 week [7 weeks headers, 1week merkleblocks]). But lets look at it like this: for an additional, say 25MB per week (maybe a bit more), you get the ability to filter blocks without depending on serving peers who may compromise your financial privacy. Also, if you keep the filters, further rescans do consume the same or less bandwidth than BF BIP37. In other words: you have the chance to potentially increase privacy by consuming bandwidth in the range of a single audio podcast per week. I would say the job of protocol developers is protect users privacy where it’s possible (as a default). It’s probably a debatable point wether 25MB per week of traffic is worth a potential increase in privacy, though I absolutely think 25MB/week is an acceptable tradeoff. Saving traffic is possible by using BIP37 or stratum/electrum… but developers should make sure users are __warned about the consequences__! Additionally, it looks like, peer operators are not endless being willing to serve – for free – a CPU/disk intense service with no benefits for the network. I would question wether a decentralised form of BIP37 is sustainable in the long run (if SPV wallet provider bootstrap a net range of NODE_BLOOM peers to make it more reliable on the network would be snake-oil). > > 2) It filters blocks only. It doesn't address unconfirmed transactions. Well, unconfirmed transaction are uncertain for various reasons. BIP158 won't allow you to filter the mempool. But as soon as you are connected to the network, you may fetch tx with inv/getdata and pick out the relevant ones (causes also traffic). Unclear and probably impossible with the current BIP158 specs to fetch transactions that are not in active relay and are not in a block (mempool txns, at least this is true with the current observed relay tactics). > 3) Afaik, it enforces/encourages address re-use. This stems from the > fact that the server decides on the filter and in particular on the > false positive rate. On wallets with many addresses, a hardcoded filter > will be too blurry and thus each block will be matched. So wallets that > follow the "one address per incoming payment" pattern (e.g. HD wallets) > at some point will be forced to wrap their key chains back to the > beginning. If I'm wrong on this one please let me know. I’m probably the wrong guy to ask (haven’t made the numbers) but last time I rescanned a Core wallet (in my dev branch) with block filters (and a Core wallet has >2000 addresses by default) it fetched a low and acceptable amount of false positive blocks. (Maybe someone who made the numbers step in here.) Though, large wallets – AFAIK – also operate badly with BIP37. > > 4) The filters are not yet committed to the blockchain. Until that > happens we'd have to trust a server to provide correct filters. I wouldn’t say so. It’s on a similar level than BIP37. BIP37 is not – and can not – be committed to the blockchain. You fully trust the peer that it won’t… a) create fake unconfirmed transactions (would be the same if a BIP158 wallet would show you unconfirmed transaction) b) lies by omission (you will miss relevant transactions, eventually swipe your wallet and loose coins) IMO, the point b) is true for BIP37 and BIP158 (as long as not commited). In both cases, you can reduce the trust by comparing between peers / filter-providers. b) is conceptually solvable with a soft-fork (commitment) in BIP158 (not with BIP37). Additionally, block-filters will, very likely, be useful for other features (load/rescan an [old] wallet on a prune peer that has the filters constructed). There is probably no clear answer like „X is better than Y“. Personally I would like to see developers being more honest/transparent to users about the implications of the used filtering,... and giving them choices. Imagine a user can choose between „Electrum / BIP37 / BIP158“ depending on his needs for privacy and availability of bandwidth. Eventually also taking the future usage of this wallet (will he load old private keys, will he receive money daily, etc.) into account. Plus, full node hardware appliances that run at home (or in a trusted environment) are solving many of these issues plus adding a bunch of great features – if done right. /jonas signature.asc Description: Message signed with OpenPGP
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
Hi Justus, It might be helpful to consult the Rust implementation of BIP158 here: https://github.com/rust-bitcoin/rust-bitcoin/blob/master/src/util/bip158.rs It has a cleaner structure than Core or Neutrino, includes server and client side and passes Core's test vectors. Regards, Tamas Blummer > On Jul 22, 2019, at 17:58, Justus Ranvier via bitcoin-dev > wrote: > > Signed PGP part > On 7/22/19 12:01 AM, Matt Corallo via bitcoin-dev wrote: >> Finally, regarding alternatives, the filter-generation code for BIP >> 157/158 has been in Bitcoin Core for some time, though the P2P serving >> side of things appears to have lost any champions working on it. I >> presume one of the Lightning folks will eventually, given they appear to >> be requiring their users connect to a handful of their own servers right >> now, but if you really need it, its likely not a ton of work to pipe >> them through. > > If you want projects to adopt BIP-157/158, you'd do well to fix the > numerous errors in the specification. > > As it stands right now it is impossible to implement the protocol using > the specification because he code examples are broken to the point of > appearing intentionally sabotaged. > > > signature.asc Description: Message signed with OpenPGP ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On Tuesday 23 July 2019 14:47:18 Andreas Schildbach via bitcoin-dev wrote: > > the DNS seed infrastructure among others can easily direct > > wallets to those nodes > > Last I checked none of the seeds did. But I agree this would be nice to > have. It's supported by default in sipa's bitcoin-seeder, which many DNS seeds use. My seed also currently requires NODE_BLOOM for *any* answers returned. (But I don't guarantee that will remain forever.) > As a side note, Coinbase just announced their 30M'th wallet. I'm > convinced this massive run into custodial wallets... Light wallets are just as bad for the network as custodial wallets. > IMHO, BIP 37 is the only scaling technology that has proven successful and > could – if supported and improved rather than choked – continue to help > holding against the bitbanks trend. Light wallets are not Bitcoin scaling. They are just trusting anonymous 3rd parties, which is harmful to Bitcoin. Luke ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On 7/23/19 9:47 AM, Andreas Schildbach via bitcoin-dev wrote: > BIP 157/158 is not an alternative to BIP 37: They complement each other pretty well though. Wallets can save the deterministic GCS filters in the same way as headers, which means blocks can be re-scanned if necessary (importing new keys, etc) offline. Bloom filters are good for limiting mempool bandwidth as well as controlling the fraction of each block that is downloaded. On the BTC chain that last point doesn't matter as so much, but on chains with larger blocks I expect that clients will ultimately end up using both filter schemes for exactly that reason. 0x9B2D74E2A30A85B9.asc Description: application/pgp-keys ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On Mon, Jul 22, 2019 at 02:52:10PM -0400, Peter via bitcoin-dev wrote: > Hi, > > I believe two wallets. Andreas' Android Bitcoin wallet and BRD are > significant users of node_bloom. > > Privacy is a matter of individual choice in the current protocol. Why not > let people provide this network service? I don't see why it should be > end-of-life if it provides value. With that patch people are still free to choose to provide bloom filtering services by setting -peerbloomfilters=1 > I believe there's a network security obtained by having a large quantity of > people following the Bitcoin headers based on longest weighted chain. As a > means of nullifying potential miner initiated hard forks (like S2X). There really isn't due to sybil attacks; we already have good reason to believe that the Bitcoin network is subject to them by deanonymization/chainanalysis services. Indeed there's a good argument that creating services that are vulnerable to sybil attacks encourages them by making them succesful at something. That creates its own risks, for instance the risk that the sybil attacker will themselves screw up and cause a bunch of nodes to go offline at once. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
(Rather than replying individually, I try to address questions and add my remarks in one post.) > Finally, regarding alternatives, the filter-generation code for > BIP 157/158 has been in Bitcoin Core for some time, though the > P2P serving side of things appears to have lost any champions > working on it. BIP 157/158 is not an alternative to BIP 37: 1) It causes way too much traffic for mobile users, and likely even too much traffic for fixed lines in not so developed parts of the world. 2) It filters blocks only. It doesn't address unconfirmed transactions. 3) Afaik, it enforces/encourages address re-use. This stems from the fact that the server decides on the filter and in particular on the false positive rate. On wallets with many addresses, a hardcoded filter will be too blurry and thus each block will be matched. So wallets that follow the "one address per incoming payment" pattern (e.g. HD wallets) at some point will be forced to wrap their key chains back to the beginning. If I'm wrong on this one please let me know. 4) The filters are not yet committed to the blockchain. Until that happens we'd have to trust a server to provide correct filters. > Can you specify exactly which wallets those are? Bitcoin Wallet, Breadwallet, BISQ and many smaller ones. > the DNS seed infrastructure among others can easily direct > wallets to those nodes Last I checked none of the seeds did. But I agree this would be nice to have. > We eventually can’t expect - in the long run - that nodes provide > disk/CPU intense services for free to clients not contributing back > to the network. I'm disappointed to learn that supporting 10M wallets gets discredited down to "no contribution". Each node of course supports just a number of them, but still... > The fact that nobody cared about extending it for SW may also > underline that BIP37 is seen as a conceptual mistake and/or "low > interest in further extensions“. I suspect this is a fallacy. BIP37 doesn't need to be extended for SegWit. See Bitcoin Wallet and Breadwallet, both support SegWit and use the original BIP 37. It's true however that BIP 37 could be made simpler to work with, more future-proof and more private by simply matching output scripts. > [1] https://github.com/petertodd/bloom-io-attack This claims to be a proof of concept, but it's missing the concept. I could not find a description of the attack and why is it likely to be exploited (and why it hasn't been exploited yet). > CPU DoS attacks are also possible If such an attack exists, it should be easy to defend against. Filter is using too much CPU time → disconnect and blacklist the IP for some time. I vaguely remember the infrastructure for misbehaving peers is already present in bitcoind. As a side note, Coinbase just announced their 30M'th wallet. I'm convinced this massive run into custodial wallets is caused by the public realizing around Dec 2017 that Bitcoin fails to scale. IMHO, BIP 37 is the only scaling technology that has proven successful and could – if supported and improved rather than choked – continue to help holding against the bitbanks trend. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On Monday 22 July 2019 18:52:10 Peter via bitcoin-dev wrote: > Privacy is a matter of individual choice in the current protocol. Why not > let people provide this network service? I don't see why it should be > end-of-life if it provides value. It's not EOL, just disabled by default. Anyone can provide it by choice. Same as with Stratum/Electrum right now (except easier to enable). > I believe there's a network security obtained by having a large quantity of > people following the Bitcoin headers based on longest weighted chain. As a > means of nullifying potential miner initiated hard forks (like S2X). This is incorrect. Such wallets strictly degrade security, as they are blindly trusting miners. They make the network more VULNERABLE to 2X-like attacks. Luke ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
People are allowed the choice, it's a change of default only. On Mon, Jul 22, 2019 at 4:41 PM Peter via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Hi, > > I believe two wallets. Andreas' Android Bitcoin wallet and BRD are > significant users of node_bloom. > > Privacy is a matter of individual choice in the current protocol. Why not > let people provide this network service? I don't see why it should be > end-of-life if it provides value. > > I believe there's a network security obtained by having a large quantity > of people following the Bitcoin headers based on longest weighted chain. As > a means of nullifying potential miner initiated hard forks (like S2X). > > Respectfully, > Peter > ___ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
Hi, I believe two wallets. Andreas' Android Bitcoin wallet and BRD are significant users of node_bloom. Privacy is a matter of individual choice in the current protocol. Why not let people provide this network service? I don't see why it should be end-of-life if it provides value. I believe there's a network security obtained by having a large quantity of people following the Bitcoin headers based on longest weighted chain. As a means of nullifying potential miner initiated hard forks (like S2X). Respectfully, Peter ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On Sunday 21 July 2019 22:56:33 Andreas Schildbach via bitcoin-dev wrote: > An estimated 10+ million wallets depend on that NODE_BLOOM to be > updated. Where do you see this number? I think it would be useful to chart. > So far, I haven't heard of an alternative, except reading all > transactions and full blocks. Electrum has a JSON-based protocol that can provide information much more efficiently. Currently, this does require nodes that run additional software and/or indexes, but there are plenty of them available, and there are steps that can be taken to improve that situation. > > It is not anticipated that > > this will result in a significant lack of availability of > > NODE_BLOOM-enabled nodes in the coming years > > Why don't you anticipate that? People almost never change defaults, > especially if it's not for their own immediate benefit. At the same > time, release notes in general recommend updating to the latest version. > I *do* anticipate this will reduce the number of nodes usable by a large > enough amount so that the feature will become unstable. Many users run older versions, and do not update immediately. Today, only 42% of listening nodes are using 0.18. (If it helps ease the concern, we can keep bloom enabled by default in Knots longer.) > I strongly recommend postponing this change until an alternative exists > and then give developers enough time to implement, test and roll out. There will be time to do so, since the functionality won't disappear overnight. Luke ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On Monday 22 July 2019 13:25:25 Jonas Schnelli via bitcoin-dev wrote: > > I also think as long as we don't have an alternative, we should improve > > the current filtering for segwit. E.g. testing the scripts themselves > > and each scriptPubKey spent by any input against the filter would do, > > and it also fixes the main privacy issue with server-side filtering > > (wallets have to add two items per address to the filter). > > I think the consensus among protocol developers is (please speak up), that > BIP37 (public server based tx filtering) – in general – was a conceptual > mistake. Maybe extending it further is the wrong step, especially when > promising alternatives like BIP158 (neutrino) are around. Neutrino is very controversial, and NOT less trustful than bloom filters. It also uses significantly more bandwidth. It seems a better approach is to add Stratum (Electrum) support, and to limit usage of all pseudo-SPV protocols to trusted peers. > The fact that nobody cared about extending it for SW may also underline that > BIP37 is seen as a conceptual mistake and/or "low interest in further > extensions“. Eric Lombrozo added segwit support. While it was never reviewed for Core, it has been included and supported in Knots since v0.15.1. As I understand it, his mSIGNA wallet also makes usage of the feature. Luke ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On 7/22/19 12:01 AM, Matt Corallo via bitcoin-dev wrote: > Finally, regarding alternatives, the filter-generation code for BIP > 157/158 has been in Bitcoin Core for some time, though the P2P serving > side of things appears to have lost any champions working on it. I > presume one of the Lightning folks will eventually, given they appear to > be requiring their users connect to a handful of their own servers right > now, but if you really need it, its likely not a ton of work to pipe > them through. If you want projects to adopt BIP-157/158, you'd do well to fix the numerous errors in the specification. As it stands right now it is impossible to implement the protocol using the specification because he code examples are broken to the point of appearing intentionally sabotaged. 0x9B2D74E2A30A85B9.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
Has someone built an analysis of how much extra bandwidth CFB uses over bloom filters? Obviously an active merchant in an impoverished country paying data rates per MB will never be able to afford CFB — so those people are being cut out of Bitcoin entirely. I suppose the plan is they will rely on custodial services now? But if someone receives say, 5 tx a day, how much more bandwidth precisely will CFB require over bloom? On Mon, Jul 22, 2019 at 8:10 AM Tom Harding via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > On 7/20/19 10:46 AM, Matt Corallo via bitcoin-dev wrote: > > (less trustful and privacy-violating) alternative > > over the coming years. > > The same paper that established the 'privacy-violating' conventional > wisdom presented mitigations which have seen little exploration. > https://eprint.iacr.org/2014/763.pdf > > Meanwhile we have custodial LN, the L-BTC altcoin and, today, a massive > push into infrastructure for fully custodial accounts. > > > ___ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On 7/20/19 10:46 AM, Matt Corallo via bitcoin-dev wrote: (less trustful and privacy-violating) alternative over the coming years. The same paper that established the 'privacy-violating' conventional wisdom presented mitigations which have seen little exploration. https://eprint.iacr.org/2014/763.pdf Meanwhile we have custodial LN, the L-BTC altcoin and, today, a massive push into infrastructure for fully custodial accounts. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
Hi Andreas
>> well-known DoS vectors
>
> I asked many people, even some "core developers" at meetings, but nobody
> ever was able to explain the DoS vector. I think this is just a myth.
No. They are not a myth [1] [2] [3].
> Yes, you can set an overly blurry filter and thus cause useless traffic,
> but it never exceeds just drinking from the full firehose (which this
> change doesn't prohibit). So where is the point? An attacker will just
> switch filtering off, or in fact has never used it.
I guess it’s not about traffic DoS. It’s about asking a peer for extensive CPU
and disk work. The NODE_BLOOM peers do provide this service for free while it’s
not directly beneficial for the Bitcoin Network (pure consumed CPU/disk time).
>
>> It is not anticipated that
>> this will result in a significant lack of availability of
>> NODE_BLOOM-enabled nodes in the coming years
>
> Why don't you anticipate that? People almost never change defaults,
> especially if it's not for their own immediate benefit. At the same
> time, release notes in general recommend updating to the latest version.
> I *do* anticipate this will reduce the number of nodes usable by a large
> enough amount so that the feature will become unstable.
I guess this is speculation.
A quick lookup in my crawler databases shows me that there are more than 8’000
„good“ peers supporting NODE_BLOOM right now.
I don’t expect that this number drops rapidly, but maybe in the long run ("in
years“, but again: speculation).
We eventually can’t expect - in the long run - that nodes provide disk/CPU
intense services for free to clients not contributing back to the network.
However, sadly, due to the privacy leaks in BIP37, I expect that there will
always be a wide range of peers offering NODE_BLOOM in return of collecting
valuable data.
>
>> clients
>> which rely on the availability of NODE_BLOOM-supporting nodes on the
>> P2P network should consider the process of migrating
>> to a more modern (and less trustful and privacy-violating) alternative
>> over the coming years.
>
> There is no such alternative.
>
> I strongly recommend postponing this change until an alternative exists
> and then give developers enough time to implement, test and roll out.
NODE_BLOOM was added in Core 0.12 [4].
BIP111 is from 2015 [2].
One who follows the protocol development should have known that defaulting
NODE_BLOOM to off was something anticipated by most developers.
I would say that there are possible alternatives, like BIP158 (though BIP157 is
not widely available on the network yet).
Client side filtering works also by collecting the filter form a centralised
service by the wallet provider(s) or a CDN.
You may miss transactions by a dishonest filter-packet, so may you by talking
to a dishonest NODE_BLOOM peer.
Of course BIP158 is still young and – who knows – eventually once committed to
consensus layer (coinbase).
> I also think as long as we don't have an alternative, we should improve
> the current filtering for segwit. E.g. testing the scripts themselves
> and each scriptPubKey spent by any input against the filter would do,
> and it also fixes the main privacy issue with server-side filtering
> (wallets have to add two items per address to the filter).
I think the consensus among protocol developers is (please speak up), that
BIP37 (public server based tx filtering) – in general – was a conceptual
mistake.
Maybe extending it further is the wrong step, especially when promising
alternatives like BIP158 (neutrino) are around.
The fact that nobody cared about extending it for SW may also underline that
BIP37 is seen as a conceptual mistake and/or "low interest in further
extensions“.
/jonas
[1] https://github.com/petertodd/bloom-io-attack
[2] https://github.com/bitcoin/bips/blob/master/bip-0111.mediawiki
[3] https://github.com/bitcoin/bitcoin/pull/9238
[4]
https://github.com/bitcoin/bitcoin/blob/47d981e8273804a040d71665a4cb16038d6717e1/doc/release-notes/release-notes-0.12.0.md#node_bloom-service-bit
signature.asc
Description: Message signed with OpenPGP
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
On Mon, Jul 22, 2019 at 12:56:33AM +0200, Andreas Schildbach via bitcoin-dev wrote: > An estimated 10+ million wallets depend on that NODE_BLOOM to be > updated. So far, I haven't heard of an alternative, except reading all > transactions and full blocks. Can you specify exactly which wallets those are? Secondly, this doesn't stop people from running NODE_BLOOM nodes, and the DNS seed infrastructure among others can easily direct wallets to those nodes. This is of course not very secure, but bloom filters have never been very secure. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: PGP signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
Hey Andreas, I think maybe some of the comments here were misunderstood - I don't anticipate that most people will change their defaults, indeed, but given the general upgrade cycles we've seen on the network over the entire course of Bitcoin's history, there's little reason to believe that many nodes with NODE_BLOOM publicly accessible will be around for at least three or four years to come, though obviously any conscious effort by folks who need those services to run nodes could extend that significantly. As for the DoS issues, a super old Proof-of-Concept of the I/O variant is here: https://github.com/petertodd/bloom-io-attack though CPU DoS attacks are also possible that use high hash counts to fill a node's CPU usage (you can pretty trivially see when a bloom-based peer connects to you just by looking at top...). Finally, regarding alternatives, the filter-generation code for BIP 157/158 has been in Bitcoin Core for some time, though the P2P serving side of things appears to have lost any champions working on it. I presume one of the Lightning folks will eventually, given they appear to be requiring their users connect to a handful of their own servers right now, but if you really need it, its likely not a ton of work to pipe them through. Matt On 7/21/19 10:56 PM, Andreas Schildbach via bitcoin-dev wrote: > An estimated 10+ million wallets depend on that NODE_BLOOM to be > updated. So far, I haven't heard of an alternative, except reading all > transactions and full blocks. > > It goes without saying pulling the rug under that many wallets is a > disastrous idea for the adoption of Bitcoin. > >> well-known DoS vectors > > I asked many people, even some "core developers" at meetings, but nobody > ever was able to explain the DoS vector. I think this is just a myth. > > Yes, you can set an overly blurry filter and thus cause useless traffic, > but it never exceeds just drinking from the full firehose (which this > change doesn't prohibit). So where is the point? An attacker will just > switch filtering off, or in fact has never used it. > >> It is not anticipated that >> this will result in a significant lack of availability of >> NODE_BLOOM-enabled nodes in the coming years > > Why don't you anticipate that? People almost never change defaults, > especially if it's not for their own immediate benefit. At the same > time, release notes in general recommend updating to the latest version. > I *do* anticipate this will reduce the number of nodes usable by a large > enough amount so that the feature will become unstable. > >> clients >> which rely on the availability of NODE_BLOOM-supporting nodes on the >> P2P network should consider the process of migrating >> to a more modern (and less trustful and privacy-violating) alternative >> over the coming years. > > There is no such alternative. > > I strongly recommend postponing this change until an alternative exists > and then give developers enough time to implement, test and roll out. > > I also think as long as we don't have an alternative, we should improve > the current filtering for segwit. E.g. testing the scripts themselves > and each scriptPubKey spent by any input against the filter would do, > and it also fixes the main privacy issue with server-side filtering > (wallets have to add two items per address to the filter). > > ___ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Bitcoin Core to disable Bloom-based Filtering by default
An estimated 10+ million wallets depend on that NODE_BLOOM to be updated. So far, I haven't heard of an alternative, except reading all transactions and full blocks. It goes without saying pulling the rug under that many wallets is a disastrous idea for the adoption of Bitcoin. > well-known DoS vectors I asked many people, even some "core developers" at meetings, but nobody ever was able to explain the DoS vector. I think this is just a myth. Yes, you can set an overly blurry filter and thus cause useless traffic, but it never exceeds just drinking from the full firehose (which this change doesn't prohibit). So where is the point? An attacker will just switch filtering off, or in fact has never used it. > It is not anticipated that > this will result in a significant lack of availability of > NODE_BLOOM-enabled nodes in the coming years Why don't you anticipate that? People almost never change defaults, especially if it's not for their own immediate benefit. At the same time, release notes in general recommend updating to the latest version. I *do* anticipate this will reduce the number of nodes usable by a large enough amount so that the feature will become unstable. > clients > which rely on the availability of NODE_BLOOM-supporting nodes on the > P2P network should consider the process of migrating > to a more modern (and less trustful and privacy-violating) alternative > over the coming years. There is no such alternative. I strongly recommend postponing this change until an alternative exists and then give developers enough time to implement, test and roll out. I also think as long as we don't have an alternative, we should improve the current filtering for segwit. E.g. testing the scripts themselves and each scriptPubKey spent by any input against the filter would do, and it also fixes the main privacy issue with server-side filtering (wallets have to add two items per address to the filter). ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
