Re: [Bitcoin-development] Integration testing for BitCoin
Hey guys, it sounds great. I read through the bitcoinj documentation and started reading the code. A few years ago it wasn't a full client, but now that I see that it's almost there, it looks much more interesting :-) Testing the reorg looks critical. Thanks for the help everyone, Adam On Sat, Apr 6, 2013 at 7:21 AM, Mike Hearn m...@plan99.net wrote: In bitcoinj we desperately need integration tests to exercise the wallet code, and I think if it was done well the tests would be applicable to bitcoind as well. There have been a series of bugs in bitcoinj that boiled down to the unit tests were not realistic enough, either because they stopped simulating too early or they weren't combining multiple different things together in the same ways as happens on the real network. Sometimes timing was an issue too. Examples of what I mean - ensure that re-orgs are handled correctly and update the wallet properly in every case, etc. Something else that would be really useful, a standalone tool that stress-tests the system. If we had a tool that randomly generated chains of transactions we might have caught the bdb lock limit bug earlier. You could write such a tool using bitcoinj easily, or the raw transaction APIs on bitcoind. On Fri, Apr 5, 2013 at 8:29 PM, Adam Ritter arit...@gmail.com wrote: Thanks guys, it sounds great. Testing the JSON-RPC is/was not the main goal, just an interface for testing. I didn't know that the bitcoinj implementation is getting close to a full implementation..it sounds interesting, as it's much easier to understand and work with. I'll look at the test cases. Thanks very much, Adam On Fri, Apr 5, 2013 at 12:42 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Fri, Apr 5, 2013 at 10:24 AM, Adam Ritter arit...@gmail.com wrote: Hey guys, I just bought some BitCoins after being lazy to do it for the last few years, but also looked at the client code and the messages that are going on this mailing list. I saw that there are quite some unit tests, but I didn't find integration test for BitCoin, and I believe that it's quite important for the future of BitCoin (making the current code more stable, testing attack scenarios, refactoring and extending code). [...] Tests that simulate multiple bitcoin users and can verify that the whole network of bitcoin clients work together to achieve the goals of Bitcoin. Also maybe [System testing](http://en.wikipedia.org/wiki/System_testing) would be a better name for the tests, but I'm not sure. I prefer to call them system tests. We use a system called blocktester that Matt Corallo wrote, https://code.google.com/r/bluemattme-bitcoinj/source/browse/core/src/test/java/com/google/bitcoin/core/FullBlockTestGenerator.java?name=fullverifr=874c5904b12d1fcec5b556429cf208f63cd4e1bc It's based on BitcoinJ and works by simulating a peer against a slightly instrumented copy of Bitcoin(d/-qt) (modified to avoid computationally expensive mining). The tests simulates many complicated network scenarios and tests the boundaries of many (hopefully all) the particular rules of the blockchain validation protocol. We can use these tests to compare different versions of the reference software to each other and to bitcoinj (or other full node implementations) as well as comparing them to our abstract understanding of what we believe the rules of the protocol to be. These tests are run as part of the automated tests on every proposed patch to the reference software. Via a robot called pulltester which comments on github requests and produces logs like this: http://jenkins.bluematt.me/pull-tester/92a129980fb9b506da6c7f876aa8adb405c88e17/. Pulltester also performs automatic code coverage measurements. Additionally, we run a public secondary test bitcoin network called 'testnet', which can be accessed by anyone by starting the reference software with testnet=1. Testnet operates the same as the production network except it allows mining low difficulty blocks to prevent it going for long times without blocks, and some of the protective relaying rules against non standard transaction types are disabled. Most of this testing work has been centered around validating the blockchain behavior because thats what has serious systemic risk. Measuring the json rpc behavior is strictly less interesting, though interesting too. -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net
Re: [Bitcoin-development] DOS-Attacks on bitcoin-client?
On Sun, Apr 7, 2013 at 10:43 AM, Oliver Egginger bitc...@olivere.de wrote: Hello, I'm using your bitcoin-qt client (version 0.8.1). Normally everything is working pretty fine, but sometimes it seems that other nodes produce an enormous amount of traffic. I have not had the time to investigate thoroughly yet. I only have briefly viewed with tshark. So far I have just restarted the client in the hope that it no longer connects with the 'evil' node. This usually works quite well. Is anything about DOS-Attacks known to you? Many new users have started using the reference client which downloads the whole blockchain from peers. There currently isn't a throttling mechanism [1] so it's possible to quickly eat up your bandwidth. You can try QoS on your router or use the -nolisten command line flag. You will still relay transactions, just not serve the whole blockchain. [1] https://github.com/bitcoin/bitcoin/issues/273 -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Who is creating non-DER signatures?
(cross-post from bitcointalk.org) Hello all, as some may know, Bitcoin uses DER-encoded signatures in its transactions. However, OpenSSL (which is used to verify them) accepts more than just the strict DER specification (it allows negative numbers, extra zero padding, extra bytes at the end, and perhaps more). As we don't like the de-facto specification of the Bitcoin block validity rules to depend on OpenSSL, we're trying to introduce a rule to make such non-standard signatures invalid. Obviously, that can't be done as long as any significant amount of clients on the network is creating these. I've monitored all transactions the past weeks (1.4M transactions), and it seems 9641 of them contain at least one non-standard signature. See https://bitcointalk.org/index.php?topic=169620.0 for a list of the top addresses that had coins used as inputs in such transactions. If you recognize any of these addresses, or have an idea of who owns them or what software they are using, please let me know. Thanks! -- Pieter -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Who is creating non-DER signatures?
It'd help to know how the signatures are invalid. On Sun, Apr 7, 2013 at 5:34 PM, Pieter Wuille pieter.wui...@gmail.comwrote: (cross-post from bitcointalk.org) Hello all, as some may know, Bitcoin uses DER-encoded signatures in its transactions. However, OpenSSL (which is used to verify them) accepts more than just the strict DER specification (it allows negative numbers, extra zero padding, extra bytes at the end, and perhaps more). As we don't like the de-facto specification of the Bitcoin block validity rules to depend on OpenSSL, we're trying to introduce a rule to make such non-standard signatures invalid. Obviously, that can't be done as long as any significant amount of clients on the network is creating these. I've monitored all transactions the past weeks (1.4M transactions), and it seems 9641 of them contain at least one non-standard signature. See https://bitcointalk.org/index.php?topic=169620.0 for a list of the top addresses that had coins used as inputs in such transactions. If you recognize any of these addresses, or have an idea of who owns them or what software they are using, please let me know. Thanks! -- Pieter -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Who is creating non-DER signatures?
On Sun, Apr 07, 2013 at 06:01:13PM +0200, Mike Hearn wrote: It'd help to know how the signatures are invalid. The majority (~90%) is negative R or S values (which are just interpreted as unsigned by OpenSSL, but if the top byte has its highest bit set, it must be preceeded by a 0x00 accordinging to DER). A smaller number uses excessively padded R or S value (with a 0x00 in front when it's not necessary). Finally there are 4 signatures with an incorrect length marker in the beginning (which likely means they contain some garbage at the end). -- Pieter -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] DOS-Attacks on bitcoin-client?
Am 07.04.2013 17:22, schrieb Scott Howard: Many new users have started using the reference client which downloads the whole blockchain from peers. Yes I have made a clean start because of the the new database structure. There currently isn't a throttling mechanism [1] so it's possible to quickly eat up your bandwidth. You can try QoS on your router or use the -nolisten command line flag. You will still relay transactions, just not serve the whole blockchain. I see. I successfully have downloaded the Blockchain again. Thus, it should not occur again now. If it does, I'll be back again. :-) Thank you for your quick help. regards Oliver -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] DOS-Attacks on bitcoin-client?
BTW, check out the blockchain torrent, as one way of offloading some of the download bandwidth used from the P2P network: Bitcoin blockchain data torrent https://bitcointalk.org/index.php?topic=145386.0 -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
