[Bitcoin-development] LevelDB in master
Now-merged pull request #2702 appears to have put the master branch on an unofficial Ripple fork of LevelDB, rather than merely updating us to LevelDB 1.12.0. While Vinnie did somewhat disclose this, I don't see any evidence the nature of this was fully understood by others. As I understood the pull request, the Ripple and Bitcoin fork was just LevelDB with the changes we had already made. Mike's comments on the pull request (his audit) suggest that this may have been the case in an earlier revision of it. But in fact, there appear to be a number of other changes included in what was finally merged a few weeks ago. Furthermore, Ripple's fork did not do a proper git merge of upstream, thus there is a break in git history, and, more importantly, a number of upstream fixes (including some we have had reported to the Bitcoin issue tracker) were not included in this merge. I've pushed three branches to https://github.com/luke-jr/leveldb : bitcoin-1.5 Our old/unreleased LevelDB 1.5 fork, for reference bitcoin Our LevelDB 1.7 fork, included in 0.8.x bitcoin-upOur LevelDB 1.7 fork, merged with upstream LevelDB 1.12 A diff from current master (Ripple LevelDB 1.12 fork) to bitcoin-up: https://gist.github.com/luke-jr/6248543 Thoughts? Luke signature.asc Description: This is a digitally signed message part. -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 32.5
I filed a bug in the bitcoinj tracker for this a few days ago referencing rfc 6967, but that RFC is very complicated and I'm not sure it's really necessary to go that far. H(sighash||key) is easy to implement and I feel I understand it better. In our case it wouldn't have helped anyway - if anything it would just delayed discovery of the underlying weakness. The same RNG is typically used to generate both keys and signatures today. However in future it may be the case that people put more effort into generating a really random key because they only have to do it once, and then the signing RNG would be different. Your concern about hardware devices leaking private key bits via a side channel is also well made, although I think you have to find some way to establish trust in these devices anyway as sniffing all their IO traffic and analysing it is really hard (plus it inverts the threat model - if you trust your computer and not your hardware wallet, why do you have a hardware wallet?) The other advantage is that deterministic keys and signatures together mean two instances of the same wallet generate identical transactions given an identical sequence of commands. This could help keep wallets in sync. For example we had a few users who got confused because they had cloned their Android wallets across devices (NOT SUPPORTED!) and then one device updated first, did key rotation, and then the other device showed a transaction that sent all their money to a new address it knew nothing about. If they didn't realise the other device had updated this looked identical to theft! I don't think fractional BIP numbers are the way to go :) but a new BIP that standardised a way to select K would, if reviewed, be something I'd implement. On Fri, Aug 16, 2013 at 4:26 AM, Gregory Maxwell gmaxw...@gmail.com wrote: I am wondering if we shouldn't have a BIP32 addendum which makes the following signing related recommendations: (1) Recommend a specific deterministic DSA derandomization procedure (a deterministic way to generate the DSA nonce), presumably one based on HMAC-SHA512 (since BIP32 uses that construct) or SHA256 in the style of RFC 6979. DSA systems being compromised due to poor randomness at runtime is not new. It effected other systems before it effected Bitcoin systems, it's not a new problem and it's not going away. It's difficult to tell if an implementation is correct or not. Use of a fully deterministic signature would allow for complete test vectors in signing and complete confidence that there is no random number related weakness in a signing implementation. In particular, with relevance to our ecosystem a maliciously modified difficult to audit hardware wallet could be leaking its keys material via its signatures. Even without producing insecure K values it could use the choice of K to leak a couple bits of an encrypted root key with every signature, and allow the malicious party to recover the keys by simply observing the network. Making the signatures deterministic would make this kind of misbehavior practically discoverable. We wouldn't be alone in making this change, in general industry is moving in this direction because it has become clear that DSA is a hazard otherwise. The primary arguments in most spaces against derandomizing DSA are FIPS conformance (irrelevant for us) and reasonable concerns about the risks of using a (less) reviewed cryptographic construct. With widespread motion towards derandomized DSA this latter concern is less of an issue. Libcrypt has also implemented derandomized DSA in git. The ed25519 signature system of DJB, et. al. also uses a similar derandomization. An alternative is implementing a still random construct where K is some H(message||key||random) which should remain secure even where the randomness is poor, but this loses the advantage of being able to externally verify that an implementation is not leaking information. OpenSSL development has implemented a form of this recently. See also: http://tools.ietf.org/rfc/rfc6979.txt (2) Recommends a procedure for using only even S values in signatures, eliminating this source of mutability in transactions. This can be accomplished via post-processing of existing signatures, but since it requires bignum math it is usually preferable to implement it along with signing. I believe someday this will become a network requirement for Bitcoin, but regardless it makes sense to implement it as a best practice sooner rather than later. Thoughts? -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
Cool. Maybe it's time for another development update on the foundation blog? On Fri, Aug 16, 2013 at 3:00 AM, Gavin Andresen gavinandre...@gmail.comwrote: Mike asked what non-0.9 code I'm working on; the three things on the top of my list are: 1) Smarter fee handling on the client side, instead of hard-coded fees. I was busy today generating scatter-plots and histograms of transaction fees versus priorities to get some insight into what miner policies look like right now. 2) First double-spend relaying and alerting, to better support low-value in-person transactions. Related: *Have *a *Snack*, Pay with *Bitcoins*http://www.tik.ee.ethz.ch/file/848064fa2e80f88a57aef43d7d5956c6/P2P2013_093.pdf 3) Work on 2-3 whitepapers on why we need to increase or remove the 1MB block size limit, how we can do it safely, and go through all of the arguments that have been made against it and explain why they're wrong. -- -- Gavin Andresen -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
The only other thing I'd like to see there is the start of a new anti-DoS framework. I think once the outline is in place other people will be able to fill it in appropriately. But the current framework has to be left behind. If I had to choose one thing to evict to make time for that, it'd be the whitepapers. At the moment we still have plenty of headroom in block sizes, even post April. It can probably be safely delayed for a while. On Fri, Aug 16, 2013 at 2:11 PM, Mike Hearn m...@plan99.net wrote: Cool. Maybe it's time for another development update on the foundation blog? On Fri, Aug 16, 2013 at 3:00 AM, Gavin Andresen gavinandre...@gmail.comwrote: Mike asked what non-0.9 code I'm working on; the three things on the top of my list are: 1) Smarter fee handling on the client side, instead of hard-coded fees. I was busy today generating scatter-plots and histograms of transaction fees versus priorities to get some insight into what miner policies look like right now. 2) First double-spend relaying and alerting, to better support low-value in-person transactions. Related: *Have *a *Snack*, Pay with *Bitcoins*http://www.tik.ee.ethz.ch/file/848064fa2e80f88a57aef43d7d5956c6/P2P2013_093.pdf 3) Work on 2-3 whitepapers on why we need to increase or remove the 1MB block size limit, how we can do it safely, and go through all of the arguments that have been made against it and explain why they're wrong. -- -- Gavin Andresen -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 32.5
On Fri, Aug 16, 2013 at 01:32:39PM +0200, Mike Hearn wrote: and analysing it is really hard (plus it inverts the threat model - if you trust your computer and not your hardware wallet, why do you have a hardware wallet?) Myself I would trust neither the hardware wallet nor my computer... So looks like the first version of the TREZOR won't support multisig - how far away are we from support? What about other manufacturers? P2SH support is probably going to be a major sticking point. The payment protocol is all well and good, but it doesn't (yet) help getting money to the individual. bitcoinj P2SH support for sending would be a major help here - lots of person-to-person trades happen via Android wallets. -- 'peter'[:-1]@petertodd.org 000b9656276a0fdab028ca759c3fd7f951fb20196c264b5cd1ce signature.asc Description: Digital signature -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
https://togami.com/~warren/archive/2013/example-bitcoind-dos-mitigation-via-iptables.txt *Anti-DoS Low Hanging Fruit: source IP or subnet connection limits* If you disallow the same IP and/or subnet from establishing too many TCP connections with your node, it becomes more expensive for attackers to use a single host exhaust a target node's resources. This iptables firewall based example has almost zero drawbacks, but it is too complicated for most people to deploy. Yes, there is a small chance that you will block legitimate connections, but there are plenty of other nodes for random connections to choose from. Configurable per source IP and source subnet limits with sane defaults enforced by bitcoind itself would be a big improvement over the current situation where one host address can consume limited resources of many target nodes. This doesn't remove the risk of a network-wide connection exhaustion attack by a determined attacker, but it at least makes multiple types of attacks a lot more expensive. This also doesn't do much against the io vulnerability, which would require major redesigns to prevent in Bitcoin. https://github.com/litecoin-project/litecoin/commit/db4d8e21d99551bef4c807aa1534a074e4b7964d *Want to safely delay the block size limit increase for another year or two? * This patch alone enables that. On Fri, Aug 16, 2013 at 2:24 AM, Mike Hearn m...@plan99.net wrote: The only other thing I'd like to see there is the start of a new anti-DoS framework. I think once the outline is in place other people will be able to fill it in appropriately. But the current framework has to be left behind. If I had to choose one thing to evict to make time for that, it'd be the whitepapers. At the moment we still have plenty of headroom in block sizes, even post April. It can probably be safely delayed for a while. On Fri, Aug 16, 2013 at 2:11 PM, Mike Hearn m...@plan99.net wrote: Cool. Maybe it's time for another development update on the foundation blog? On Fri, Aug 16, 2013 at 3:00 AM, Gavin Andresen gavinandre...@gmail.comwrote: Mike asked what non-0.9 code I'm working on; the three things on the top of my list are: 1) Smarter fee handling on the client side, instead of hard-coded fees. I was busy today generating scatter-plots and histograms of transaction fees versus priorities to get some insight into what miner policies look like right now. 2) First double-spend relaying and alerting, to better support low-value in-person transactions. Related: *Have *a *Snack*, Pay with *Bitcoins*http://www.tik.ee.ethz.ch/file/848064fa2e80f88a57aef43d7d5956c6/P2P2013_093.pdf 3) Work on 2-3 whitepapers on why we need to increase or remove the 1MB block size limit, how we can do it safely, and go through all of the arguments that have been made against it and explain why they're wrong. -- -- Gavin Andresen -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
A ban-subnet RPC would be a reasonable addition, but obviously DoS attackers that are IP or bandwidth constrained are really just script kiddies. Also anything that involves every node operator doing manual intervention rather works against decentralisation and having a big network. That's why I keep pushing for automated heuristic driven prioritisation. On Fri, Aug 16, 2013 at 3:41 PM, Warren Togami Jr. wtog...@gmail.comwrote: https://togami.com/~warren/archive/2013/example-bitcoind-dos-mitigation-via-iptables.txt *Anti-DoS Low Hanging Fruit: source IP or subnet connection limits* If you disallow the same IP and/or subnet from establishing too many TCP connections with your node, it becomes more expensive for attackers to use a single host exhaust a target node's resources. This iptables firewall based example has almost zero drawbacks, but it is too complicated for most people to deploy. Yes, there is a small chance that you will block legitimate connections, but there are plenty of other nodes for random connections to choose from. Configurable per source IP and source subnet limits with sane defaults enforced by bitcoind itself would be a big improvement over the current situation where one host address can consume limited resources of many target nodes. This doesn't remove the risk of a network-wide connection exhaustion attack by a determined attacker, but it at least makes multiple types of attacks a lot more expensive. This also doesn't do much against the io vulnerability, which would require major redesigns to prevent in Bitcoin. https://github.com/litecoin-project/litecoin/commit/db4d8e21d99551bef4c807aa1534a074e4b7964d *Want to safely delay the block size limit increase for another year or two?* This patch alone enables that. On Fri, Aug 16, 2013 at 2:24 AM, Mike Hearn m...@plan99.net wrote: The only other thing I'd like to see there is the start of a new anti-DoS framework. I think once the outline is in place other people will be able to fill it in appropriately. But the current framework has to be left behind. If I had to choose one thing to evict to make time for that, it'd be the whitepapers. At the moment we still have plenty of headroom in block sizes, even post April. It can probably be safely delayed for a while. On Fri, Aug 16, 2013 at 2:11 PM, Mike Hearn m...@plan99.net wrote: Cool. Maybe it's time for another development update on the foundation blog? On Fri, Aug 16, 2013 at 3:00 AM, Gavin Andresen gavinandre...@gmail.com wrote: Mike asked what non-0.9 code I'm working on; the three things on the top of my list are: 1) Smarter fee handling on the client side, instead of hard-coded fees. I was busy today generating scatter-plots and histograms of transaction fees versus priorities to get some insight into what miner policies look like right now. 2) First double-spend relaying and alerting, to better support low-value in-person transactions. Related: *Have *a *Snack*, Pay with *Bitcoins*http://www.tik.ee.ethz.ch/file/848064fa2e80f88a57aef43d7d5956c6/P2P2013_093.pdf 3) Work on 2-3 whitepapers on why we need to increase or remove the 1MB block size limit, how we can do it safely, and go through all of the arguments that have been made against it and explain why they're wrong. -- -- Gavin Andresen -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes.
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
On Fri, Aug 16, 2013 at 02:24:04PM +0200, Mike Hearn wrote: The only other thing I'd like to see there is the start of a new anti-DoS framework. I think once the outline is in place other people will be able to fill it in appropriately. But the current framework has to be left behind. Part of anti-DoS should include making it easier for people to contribute back to the network. Lowest hanging fruit: 1) SPV nodes with spare bandwidth should relay whole blocks to reduce the load on full-nodes. The SPV security model is based on hashing power anyway, so there is no major harm in doing so - if you have the resources to fake blocks, you probably have the resources to sybil the network anyway. 2) It's probably reasonable for SPV peers with bandwidth to be willing to do bloom filtering on the behalf of peers that don't have spare bandwidth. Hence they would set NODE_BLOOM, but not NODE_NETWORK. (or more likely some more nuanced flags) Again this has to apply to full blocks only unless we come up with some PoW scheme or similar to discourage DoS attacks via invalid transactions. (maintaining partial UTXO sets is another possibility, although a complex one) Both examples work best with payment protocols where the recipient is responsible for getting the transaction broadcast. Similarly you can by default not connect to full-node peers, and then connect to them on demand when you have a transaction to send. Doing this also makes it more difficult to sybil the network - for instance right now you can create SPV honeypots that allow incoming connections only from SPV nodes, thus attracting a disproportionate % of the total SPV population given a relatively small number of nodes. You can then use that to harm SPV nodes by, for instance, making a % of transactions be dropped deterministicly, either by the bloom matching code, or when sent. Users unlucky enough to be surrounded by sybil nodes will have their transactions mysteriously fail to arrive in their wallets, or have their transactions mysteriously never confirm. Given how few full nodes there are, it probably won't take very many honeypots to pull off this attack, especially if you combine it with a simultaneous max connections or bloom io attack to degrade the capacity of honest nodes. Deanonymization is another attack that can be done at the same time of course. In any case, the more peers on the network that can relay data the better. 3) Make it easier to run a full node. IMO partial mode is the way to go here. Note that from a legal perspective we really want to make sure that running full nodes (and mining p2pool or solo) continue to be something ordinary users do. We do not want to give the impression to regulators that running a full node is in any way unusual or rare, and thus something that might be practical or desirable to regulate. Users should be given clear options about how much disk space and bandwidth to donate to the health of the network, and within those limits nodes should always try to make progress towards being full nodes, and in the meantime being increasingly productive partial nodes. Even with pure SPV nodes if they are relaying block data and ideally transactions they make it much more difficult for regulations to be made that, say, require full node operators to apply blacklists to transactions in the mempool or in blocks. 4) This is really a side effect, and not directly DoS-related, but unfortunately we're going to have to create some kind of proof-of-UTXO-possession that miners include in the blocks they mine. With partial mode it's just too easy and tempting for people to mine blocks with fee paying transactions in them without actually having the full UTXO set; such nodes can't tell if a block is invalid due to a fake txin, and thus will happily extend a invalid chain. This possession proof can probably be make part of a UTXO commitment. If I had to choose one thing to evict to make time for that, it'd be the whitepapers. At the moment we still have plenty of headroom in block sizes, even post April. It can probably be safely delayed for a while. Lots of off-chain tx solutions are popping up which will help push back the issue's relevance as well. Also your micropayment channels possibly. -- 'peter'[:-1]@petertodd.org 000b9656276a0fdab028ca759c3fd7f951fb20196c264b5cd1ce signature.asc Description: Digital signature -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
That change was made in response to user complaints. Heck we get complaints about battery life and bandwidth impact even with Bloom filtering. We can't just randomly start using peoples bandwidth for relaying blocks, especially as I guess most SPV nodes are behind NAT. If Gavin is right and the future is dominated by mobiles and tablets, then it will require a change of thinking in how P2P networks work. I think there are plenty of people with private servers who would be willing to run nodes though. I'm not too worried about this. On Fri, Aug 16, 2013 at 4:27 PM, Warren Togami Jr. wtog...@gmail.comwrote: bitcoinj-0.10 release notes: - We now require Bloom-capable (0.8+) peers by default and will disconnect from older nodes. This avoids accidental bandwidth saturation on mobile devices. Given the user-security concern that Peter brings up, reconsideration of this new default behavior in SPV clients may be warranted. On Fri, Aug 16, 2013 at 4:15 AM, Peter Todd p...@petertodd.org wrote: On Fri, Aug 16, 2013 at 10:01:16AM -0400, Peter Todd wrote: Doing this also makes it more difficult to sybil the network - for instance right now you can create SPV honeypots that allow incoming connections only from SPV nodes, thus attracting a disproportionate % of the total SPV population given a relatively small number of nodes. You can then use that to harm SPV nodes by, for instance, making a % of transactions be dropped deterministicly, either by the bloom matching code, or when sent. Users unlucky enough to be surrounded by sybil nodes will have their transactions mysteriously fail to arrive in their wallets, or have their transactions mysteriously never confirm. Given how few full nodes there are, it probably won't take very many honeypots to pull off this attack, especially if you combine it with a simultaneous max connections or bloom io attack to degrade the capacity of honest nodes. Oh, here's an even better way to do the tx drop attack: when you drop a transaction, make a fake one that pays the same scriptPubKeys with the same amount, and send it to the SPV peer instead. They'll see the transaction go through and show up in their wallet, but it'll look like it got stuck and never confirmed. They'll soon wind up with a wallet full of useless transactions, effectively locking them out of their money. Here's another question for you Mike: So does bitcoinj have any protections against peers flooding you with useless garbage? It'd be easy to rack up a user's data bill for instance by just creating junk unconfirmed transactions matching the bloom filter. -- 'peter'[:-1]@petertodd.org 0018dcf5bcc3f018a05517ba1c479b432ba422015d4506496e55 -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
On Fri, Aug 16, 2013 at 6:41 AM, Warren Togami Jr. wtog...@gmail.com wrote: If you disallow the same IP and/or subnet from establishing too many TCP connections with your node, [...] has almost zero drawbacks, There are whole countries who access the internet from single IP addresses. There are major institution with hundreds or even thousands of hosts that could be running Bitcoin who are visible to the public internet as a single IP address (/single subnet). Most tor traffic exits to the internet from a dozen of the largest exits, common local-network configurations have people addnode-ing local hosts from many systems on a subnet, etc. Prioritizing the availability of inbound slots based on source IP is reasonable and prudent, but it does not have almost zero drawbacks. Outright limiting is even worse. As a protective measure its also neigh useless for IPv6 connected hosts and hidden service hosts. It's also ineffective at attacks which exhaust your memory, cpu, IO, or bandwidth without trying to exhaust your sockets. So I am not opposed to prioritizing based on it (e.g. when full pick an inbound connection to drop based on criteria which includes network mask commonality), but I would not want to block completely based on this. -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
On Fri, Aug 16, 2013 at 04:36:20PM +0200, Mike Hearn wrote: That change was made in response to user complaints. Heck we get complaints about battery life and bandwidth impact even with Bloom filtering. We can't just randomly start using peoples bandwidth for relaying blocks, especially as I guess most SPV nodes are behind NAT. UPNP seems to work well for the reference client. What's the situation there on Android? I leave my phone plugged in and connected via wifi for most of the day; lots of people do that. The user interface for this stuff is very simple: How much bandwidth will you contribute back? If you contribute more bandwidth back, other peers will prioritize you and your wallet will be more reliable. -- 'peter'[:-1]@petertodd.org 003cfc051263917373a1cab2655994b97c54a625021f52c84658 signature.asc Description: Digital signature -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
On Fri, Aug 16, 2013 at 4:59 PM, Peter Todd p...@petertodd.org wrote: UPNP seems to work well for the reference client. What's the situation there on Android? Not sure - it could be investigated. I think UPNP is an entirely userspace-implementable protocol, so in theory it could be done by a userspace library (even libminiupnp - java is not a requirement on android) I leave my phone plugged in and connected via wifi for most of the day; lots of people do that. I suspect you mean I think lots of people do that. I'm not so sure. We could potentially run an experiment in the Android app to measure how many users are in a position to contribute back, but just because you have wifi doesn't mean you can reconfigure it using UPnP. That helps a lot in home networks, but at the office it doesn't help. I'm wary of a ton of work being put in to achieve not very much here. Satoshi's original vision was always that millions of users were supported by 100,000 or so nodes. I don't think that's unreasonable over the long term. Besides, prioritisation isn't very hard. Nodes can just hand clients a signed timestamp which they remember. When re-connecting, the signed timestamp is handed back to the node and it gives priority to those with old timestamps. No state is required on the node side. Signing and checking can be passed onto the general ECDSA thread pool that works its way through pending signature operations, they'd be prioritised lower than checking blocks/broadcasts. -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
On Fri, Aug 16, 2013 at 05:11:35PM +0200, Mike Hearn wrote: On Fri, Aug 16, 2013 at 4:59 PM, Peter Todd p...@petertodd.org wrote: UPNP seems to work well for the reference client. What's the situation there on Android? Not sure - it could be investigated. I think UPNP is an entirely userspace-implementable protocol, so in theory it could be done by a userspace library (even libminiupnp - java is not a requirement on android) Do find out. I leave my phone plugged in and connected via wifi for most of the day; lots of people do that. I suspect you mean I think lots of people do that. I'm not so sure. We could potentially run an experiment in the Android app to measure how many users are in a position to contribute back, but just because you have wifi doesn't mean you can reconfigure it using UPnP. That helps a lot in home networks, but at the office it doesn't help. Also worth finding out. I'm wary of a ton of work being put in to achieve not very much here. Satoshi's original vision was always that millions of users were supported by 100,000 or so nodes. I don't think that's unreasonable over the long term. Appeal to authority. Stop bringing up Satoshi's vision - our understanding of crypto-currencies has improved in the 4.5 years since Bitcoin was released. Satoshi didn't even forsee pool mining, which says a lot about his economic judgement. Besides, prioritisation isn't very hard. Nodes can just hand clients a signed timestamp which they remember. When re-connecting, the signed timestamp is handed back to the node and it gives priority to those with old timestamps. No state is required on the node side. Signing and checking can be passed onto the general ECDSA thread pool that works its way through pending signature operations, they'd be prioritised lower than checking blocks/broadcasts. Right, so you're giving priority to peers that have been around for awhile. You've succeeded in forcing attackers to wait a bit. A) What's the definition of a peer? What stops me from pretending to be 100 peers? B) Given an attacker willing to wait, what's your plan? -- 'peter'[:-1]@petertodd.org 004a52a297d9ae8ecde2ba62b681cc5a4cfbf7636032fc78e7d0 signature.asc Description: Digital signature -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 32.5
I personally like the full-measure of eliminating the CS-PRNG entirely from signing. If the random component is assumed to be untrusted, keeping it in there adds no value, while eschewing the main benefit of deterministic signing (ease of testing, auditing) This just leaves the CS-PRNG at the heart of the security system -- when generating the root master key of an HD wallet. Adding to what Mike said, a single invocation of a CS-PRNG driving all subsequent keys increases the attack value if that one invocation turns out to be weak. By comparison, at least compromised DSA signatures were one-off events which didn't allow theft of funds beyond the one compromised address. Cumulative / rolling entropy collection over time through multiple CS-PRNG invocations, or multiple entropy sources, could serve to recover from an occasionally weak CS-PRNG. I've read claims that this is bad practice because a single low entropy source can take entropy out of the result, but this seems like FUD. If you're using SHA512-HMAC to hash chain a few entropy sources, even return 4; // chosen by random dice roll is not going to help, but it's not going to hurt. The DSA 'repeated-k' basically advertises itself on the block-chain and people were actively scanning for this weakness, whereas a weak key in the BIP32 root might not be as apparent, so exploitation may be more difficult, but also more insidious. Of course this depends on the exact failure mode of the CS-PRNG being used -- I wonder if anyone is searching for BIP32 keys based off of one of the 32k Debian random numbers being used as a master key? Smartphones in particular have lots of sensors which could provide entropy. For example, if you pulled 64 bytes from secure random, you could at least HMAC that with the SHA512 of a picture or a short video sample taken by the user. I'm guessing some people would cringe at this, but it seems to me like it provides some measure of protection to justify the increased code complexity. TL;DR - Really like the idea of minimizing CS-PRNG use whenever possible (deterministic signing) and also would love to learn more best practices for placing less trust in the so called CS-PRNG when we do have to use them. Thanks, Jeremy -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
A sane default that better protects users could be... If (plugged into power) (wifi) then non-bloom peers are OK. It would protect those users more than reliance upon on the smaller subset of bloom nodes. Scale back to the less secure behavior when battery and bandwidth matters. Warren On Fri, Aug 16, 2013 at 4:36 AM, Mike Hearn m...@plan99.net wrote: That change was made in response to user complaints. Heck we get complaints about battery life and bandwidth impact even with Bloom filtering. We can't just randomly start using peoples bandwidth for relaying blocks, especially as I guess most SPV nodes are behind NAT. If Gavin is right and the future is dominated by mobiles and tablets, then it will require a change of thinking in how P2P networks work. I think there are plenty of people with private servers who would be willing to run nodes though. I'm not too worried about this. On Fri, Aug 16, 2013 at 4:27 PM, Warren Togami Jr. wtog...@gmail.comwrote: bitcoinj-0.10 release notes: - We now require Bloom-capable (0.8+) peers by default and will disconnect from older nodes. This avoids accidental bandwidth saturation on mobile devices. Given the user-security concern that Peter brings up, reconsideration of this new default behavior in SPV clients may be warranted. On Fri, Aug 16, 2013 at 4:15 AM, Peter Todd p...@petertodd.org wrote: On Fri, Aug 16, 2013 at 10:01:16AM -0400, Peter Todd wrote: Doing this also makes it more difficult to sybil the network - for instance right now you can create SPV honeypots that allow incoming connections only from SPV nodes, thus attracting a disproportionate % of the total SPV population given a relatively small number of nodes. You can then use that to harm SPV nodes by, for instance, making a % of transactions be dropped deterministicly, either by the bloom matching code, or when sent. Users unlucky enough to be surrounded by sybil nodes will have their transactions mysteriously fail to arrive in their wallets, or have their transactions mysteriously never confirm. Given how few full nodes there are, it probably won't take very many honeypots to pull off this attack, especially if you combine it with a simultaneous max connections or bloom io attack to degrade the capacity of honest nodes. Oh, here's an even better way to do the tx drop attack: when you drop a transaction, make a fake one that pays the same scriptPubKeys with the same amount, and send it to the SPV peer instead. They'll see the transaction go through and show up in their wallet, but it'll look like it got stuck and never confirmed. They'll soon wind up with a wallet full of useless transactions, effectively locking them out of their money. Here's another question for you Mike: So does bitcoinj have any protections against peers flooding you with useless garbage? It'd be easy to rack up a user's data bill for instance by just creating junk unconfirmed transactions matching the bloom filter. -- 'peter'[:-1]@petertodd.org 0018dcf5bcc3f018a05517ba1c479b432ba422015d4506496e55 -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
