Re: [Bitcoin-development] Why are we bleeding nodes?
On Wed, Apr 9, 2014 at 12:38 PM, Wendell w...@hivewallet.com wrote: On that note, I think we have every possibility to make desktop and mobile wallets mind-numbingly simple -- and perhaps even do one better. Is this now a community priority? If so, I would really appreciate some additional contributors to Hive! How does that relate to the nodes issue? Would packaging an optional bitcoind with your wallet be an option, which is automatically managed in the background, so that users can run a full node if they want? Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
YES Such a bitcoind is what I called border router in a previous mail. Yes, SPV wallets are getting ahead of features, so people will use them also because on size just does not fit all, but all want to ensure being on the same trunk of the chain. Therefore serious user of Bitcoin run a bitcoind as a border router and connect SPV wallets with higher functionality to that trusted node(s). This is what I think the core should focus on: Being a lightweight superfast consensus building border router and nothing more. No wallet, no GUI, no RPC calls, no Payment protocol and the rest. Regards, Tamas Blummer http://bitsofproof.com On 09.04.2014, at 17:29, Wladimir laa...@gmail.com wrote: Hello, This is primarily aimed at developers of SPV wallets. The recently reported decrease in number of full nodes could have several reasons, one of them that less people are running Bitcoin Core for the wallet because the other wallets are getting ahead in both features and useability. It's great to see innovation in wallets, but it's worrying that the number of full nodes decreases. It may be that lots of people would support the network by running a full node, but don't want to go through the trouble of installing bitcoin core separately (and get confused because it's a wallet, too). Hence I'd like to explore the idea of adding an option to popular SPV wallets, to spin a bitcoind process in the background. This could be pretty much transparent to the user - it would sync in the background, the wallet could show statistics about the node, but is not dependent on it. In exchange the user would get increased (full node level) security, as the SPV wallet would have a local trusted node. Does this sound like a good idea? Is there any way that Bitcoin Core can help to accomedate this 'embedded' usage? Specific Interfaces, special builds - maybe add a walletless bitcoind build to gitian - bindings, dlls, etc? Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development signature.asc Description: Message signed with OpenPGP using GPGMail -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
How would this affect the user in terms of disk storage? They're going to get hammered on space constraints aren't they? If it's not required how likely are users to enable this? On Wed, Apr 9, 2014 at 11:29 AM, Wladimir laa...@gmail.com wrote: Hello, This is primarily aimed at developers of SPV wallets. The recently reported decrease in number of full nodes could have several reasons, one of them that less people are running Bitcoin Core for the wallet because the other wallets are getting ahead in both features and useability. It's great to see innovation in wallets, but it's worrying that the number of full nodes decreases. It may be that lots of people would support the network by running a full node, but don't want to go through the trouble of installing bitcoin core separately (and get confused because it's a wallet, too). Hence I'd like to explore the idea of adding an option to popular SPV wallets, to spin a bitcoind process in the background. This could be pretty much transparent to the user - it would sync in the background, the wallet could show statistics about the node, but is not dependent on it. In exchange the user would get increased (full node level) security, as the SPV wallet would have a local trusted node. Does this sound like a good idea? Is there any way that Bitcoin Core can help to accomedate this 'embedded' usage? Specific Interfaces, special builds - maybe add a walletless bitcoind build to gitian - bindings, dlls, etc? Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 8:42 AM, Brian Hoffman brianchoff...@gmail.com wrote: How would this affect the user in terms of disk storage? They're going to get hammered on space constraints aren't they? If it's not required how likely are users to enable this? If Bitcoin core activates pruning a full node can be supported in— say— 4GBytes or so. (That gives enough space to store the utxo about 350MB now, and a couple gigs for blocks to serve out). I'd imagine getting information from SPV wallet developers how much disk usage agility they think is required is part of what Wladimir is looking for. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 8:41 AM, Natanael natanae...@gmail.com wrote: This could probably be done fairly easily by bundling Stratum (it's not just for pools!) and allowing SPV wallets to ask Bitcoind to start it (if you don't use it, there's no need to waste the resources), and then connect to it. The point of using Stratum is that it already is being used by Electrum, Sadly today Electrum requires more than a full node, it requires a number of large additional indexes over what a full node has and pruning is precluded. I don't think that increasing the resource utilization of the node is a good way to go there for the purposes expressed here. (not that electrum couldn't be used here, but not unmodified without the resource usage increasing route) and that it might be an easier way to support SPV clients than creating a new API in bitcoind for it since Stratum itself already relies on bitcoind to provide it's services. Bitcoin's own P2P protocol is already the API for a ordinary SPV client. So I don't believe any new API would be require, except perhaps for some process management stuff (which also isn't provided for Electrum). -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
Le 09/04/2014 17:54, Gregory Maxwell a écrit : Sadly today Electrum requires more than a full node, it requires a number of large additional indexes over what a full node has and pruning is precluded. I don't think that increasing the resource utilization of the node is a good way to go there for the purposes expressed here. (not that electrum couldn't be used here, but not unmodified without the resource usage increasing route) Electrum uses two large indexes: address - utxo (patricia tree, aka ultimate blockchain compression, see thread started by Alan Reiner in the bitcointalk forum) address - spent history The first index is not going to grow larger than what bitcoind already needs to store, because bitcoind will always need to store utxos. The second index threatens to become large. However, Electrum servers do not keep the full histories, they prune older entries. Without adapting Electrum clients, it would even be possible to keep only one bit per address (to know whether that address has been used or not), and that information is only used to restore wallets from seed, not during normal operations. If the first index (patricia tree) was implemented in bitcoind, that would obviously be a big relief for electrum servers. and that it might be an easier way to support SPV clients than creating a new API in bitcoind for it since Stratum itself already relies on bitcoind to provide it's services. Bitcoin's own P2P protocol is already the API for a ordinary SPV client. So I don't believe any new API would be require, except perhaps for some process management stuff (which also isn't provided for Electrum). -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
I am glad that SPV wallets are discussed outside the scope of mobile devices! Yes, SPV is a sufficient API to a trusted node to build sophisticated features not offered by the core. SPV clients of the border router will build their own archive and indices based on their interest of the chain therefore the border router core does not need to store (and process) anything not needed for consensus, its memory or disk footprint would be as low as an optimal storage of UTXO. Regards, Tamás Blummer http://bitsofproof.com On 09.04.2014, at 17:57, Gregory Maxwell gmaxw...@gmail.com wrote: On Wed, Apr 9, 2014 at 8:42 AM, Brian Hoffman brianchoff...@gmail.com wrote: How would this affect the user in terms of disk storage? They're going to get hammered on space constraints aren't they? If it's not required how likely are users to enable this? If Bitcoin core activates pruning a full node can be supported in— say— 4GBytes or so. (That gives enough space to store the utxo about 350MB now, and a couple gigs for blocks to serve out). I'd imagine getting information from SPV wallet developers how much disk usage agility they think is required is part of what Wladimir is looking for. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development signature.asc Description: Message signed with OpenPGP using GPGMail -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On 04/09/2014 09:09 AM, Tamas Blummer wrote: Yes, SPV is a sufficient API to a trusted node to build sophisticated features not offered by the core. SPV clients of the border router will build their own archive and indices based on their interest of the chain therefore the border router core does not need to store (and process) anything not needed for consensus, its memory or disk footprint would be as low as an optimal storage of UTXO. Storing zero full blocks does nothing to aid the network. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
1) It's more private. Bloom filters gives away quite accurate statistical information about what coins you own to whom ever you happen to be connected too. An attacker can easily use this to deanonymize you even if you don't reuse addresses; Tor does not help much against this attack. There is also an option to download everything, but do only a very basic surface validation (without keeping track of UTXOs). You do not need a full node for that. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Fwd: Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 5:42 PM, Brian Hoffman brianchoff...@gmail.comwrote: How would this affect the user in terms of disk storage? They're going to get hammered on space constraints aren't they? If it's not required how likely are users to enable this? If they make the (conscious) choice to run a full node they will require the bandwidth and disk space to run it. The difference with running Bitcoin Core as wallet will be that they can choose their own wallet to go with the full node. Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 7:33 PM, Alex Mizrahi alex.mizr...@gmail.com wrote: 1) It's more private. Bloom filters gives away quite accurate statistical information about what coins you own to whom ever you happen to be connected too. An attacker can easily use this to deanonymize you even if you don't reuse addresses; Tor does not help much against this attack. There is also an option to download everything, but do only a very basic surface validation (without keeping track of UTXOs). You do not need a full node for that. You may not *need* a full node, but the point of this (which I clearly explained in my opening post) would be to support the network. Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 9 April 2014 12:27:13 GMT-04:00, Tamas Blummer ta...@bitsofproof.com wrote: A border router that is not able to serve blocks is still protecting consensus rules, that SPVs do not. If the network would only consist of SPV nodes only then e.g. a majority coalition of miner could increase their reward at will. Archives need a different solution. Any collective group that has a majority of hashing power will have no major issues running enough nodes that follow their rules to make SPV insecure anyway. There's no good reason not to have SPV security nodes distribute block chain data, particularly block headers. It helps provide redundancy in the network topology and helps provide more resources for full nodes to sync up faster. For instance in a network with a large number of partial UTXO set nodes if those nodes are forwarding block data to each other they can get enough data to become fully fledged full nodes without putting all the load on the existing full nodes. This is a good thing. -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQFQBAEBCgA6BQJTRYdYMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhdwRB/46MAw7OHwnVkLHOD0g Y4X6p1/QHgRisJIgpG2Y4nGVeAjOFleQWe4PWS4Wwdr4u0BDGPmJompiR3A99CaL MUPnxJhtdiUsomn6kI704f5pkrqslQGLzejWFb7/9WuQtvGm8RwnzIs7uAqKasni KJMn3jmqFIUcCEy9dePUdmMySCQj8qSxjGDdwZnwf8BZSdSLqd8dYiN0jQi/aA1I 2hWWyyDK9V9yQ8peAg+1dfg754Tc76lj3mEQOD39Wu3Klb9mBF3+wQCW2tJYEj2E stzeOdFZsUNUIOKFb6mo0IoUipPOvrAkfm91ais+FIwlCf+k5KcwmAUXpV45rLHm egCr =2vMf -END PGP SIGNATURE- -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
Block header has to be available in SPV and also in an UTXO only storing core node, so why not serve it if bandwith allows. Serving any additional information like known peer adresses or known full blocks is certainly beneficial and should be offered if at hand. Regards, Tamas Blummer http://bitsofproof.com On 09.04.2014, at 19:46, Peter Todd p...@petertodd.org wrote: Signed PGP part On 9 April 2014 12:27:13 GMT-04:00, Tamas Blummer ta...@bitsofproof.com wrote: A border router that is not able to serve blocks is still protecting consensus rules, that SPVs do not. If the network would only consist of SPV nodes only then e.g. a majority coalition of miner could increase their reward at will. Archives need a different solution. Any collective group that has a majority of hashing power will have no major issues running enough nodes that follow their rules to make SPV insecure anyway. There's no good reason not to have SPV security nodes distribute block chain data, particularly block headers. It helps provide redundancy in the network topology and helps provide more resources for full nodes to sync up faster. For instance in a network with a large number of partial UTXO set nodes if those nodes are forwarding block data to each other they can get enough data to become fully fledged full nodes without putting all the load on the existing full nodes. This is a good thing. signature.asc Description: Message signed with OpenPGP using GPGMail -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
The right way to start with this, if anyone cares, is to add instrumentation to existing SPV wallet apps to report back to home base how long they are running for, how much disk space / RAM they have, and possibly what kind of hardware. I *strongly* suspect that the vast majority of SPV wallets are not left running permanently, and run on laptops where battery life is at a premium. These people will never want to run full nodes. Sorry. I don't think it will ever make sense to run full nodes on consumer hardware again. Our time is much better spent on optimising so it's cheaper for full node operators to run them on cheap virtualised servers. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 9 April 2014 13:50:03 GMT-04:00, Tamas Blummer ta...@bitsofproof.com wrote: Block header has to be available in SPV and also in an UTXO only storing core node, so why not serve it if bandwith allows. Serving any additional information like known peer adresses or known full blocks is certainly beneficial and should be offered if at hand. Big security advantages too. For instance if an attacker hacks, say, 10℅ of hashing power the next step for them to attack SPV clients is to try to Sybil attack them so they won't find out about the longer chain. The fewer providers of block chain data there are out there the easier that attack is - just simultaneously DoS a bunch of nodes, perhaps by a low-bandwidth exploit like the bloom io or division by zero DoS attacks. This is much harder to pull off if every SPV client is passing around block headers. Similarly by passing around full blocks the attacker has a harder time knocking other miners off the network. Regardless of whether or not a miner's peers are fully validating chain data they still have the data they need to mine the next block and thus extend the longest correct chain. -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQFQBAEBCgA6BQJTRYu+MxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwheuQCADUyClLOLP1xpG1000l uzcfPTZuIXTpzOAmYHKs/MSb6mph/Shsu0/94eW7npQNSVeZC8wQQZ1oFQ9j1GJc SKViYJfn5yMdNvMkaWazhC0r3jxxF0AI7oy2KlnSjasfczfOQuYICJadTCwvUHrb GrKVDbgsKNzZYYKn86vF4hsLwtJN4moeqX85TYN1DC7//7hgNywA73Xt2/gdwfqe LOsD4nS7mUQObQd6TcLwXDDNEGTrdS572jdYH5sykwZjPH+wqwcm2WKTnIULsJR0 OwGUi505AKJJnLcEmZ/kGbCmKB+xJ5kExjlExtcUPrJlc+xqubhnnGCMjBiGCXSY kYCK =HXRZ -END PGP SIGNATURE- -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 8:00 PM, Mike Hearn m...@plan99.net wrote: The right way to start with this, if anyone cares, is to add instrumentation to existing SPV wallet apps to report back to home base how long they are running for, how much disk space / RAM they have, and possibly what kind of hardware. I *strongly* suspect that the vast majority of SPV wallets are not left running permanently, and run on laptops where battery life is at a premium. These people will never want to run full nodes. Bitcoins stands or falls with people running full nodes. If no one wants to volunteer resources to support the network anymore, we'll have failed. Sorry. I don't think it will ever make sense to run full nodes on consumer hardware again. Our time is much better spent on optimising so it's cheaper for full node operators to run them on cheap virtualised servers. Most consumer hardware is much more powerful than 'cheap virtualized servers'. More memory, disks are cheap, and at least in the Netherlands home bandwidth is much cheaper than server bandwidth. Also: any optimization that helps running on cheap servers will also help running it on consumer hardware. It's not the one or the other. Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On 4/9/2014 11:29 AM, Wladimir wrote: Hello, This is primarily aimed at developers of SPV wallets. The recently reported decrease in number of full nodes could have several reasons, one of them that less people are running Bitcoin Core for the wallet because the other wallets are getting ahead in both features and useability. It's great to see innovation in wallets, but it's worrying that the number of full nodes decreases. It may be that lots of people would support the network by running a full node, but don't want to go through the trouble of installing bitcoin core separately (and get confused because it's a wallet, too). Hence I'd like to explore the idea of adding an option to popular SPV wallets, to spin a bitcoind process in the background. This could be pretty much transparent to the user - it would sync in the background, the wallet could show statistics about the node, but is not dependent on it. In exchange the user would get increased (full node level) security, as the SPV wallet would have a local trusted node. Does this sound like a good idea? Is there any way that Bitcoin Core can help to accomedate this 'embedded' usage? Specific Interfaces, special builds - maybe add a walletless bitcoind build to gitian - bindings, dlls, etc? Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development I personally like the ida. Are you talking about a flag that could toggle this in the background mode or recoding for in the background use? -- Kevin -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 06:19 PM, Wladimir wrote: If no one wants to volunteer resources to support the network anymore, we'll have failed. If the security of the network depends on a broken incentive model, then fix the design of the network so that economics works for you instead of against you. - -- Support online privacy by using email encryption whenever possible. Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTRZLiAAoJECoisBQbQ4v0RRMIAKL84ju4b9XAls9sOxQGrLMr xifQDhThCQKC4/qkOmGU8zseIdKRgXEq4auMF9/0BlgoSxsBcKynlRH2fFtYY7ol sdjCy/5dk+MBu3K1GCvNn/cuGkpIJJxrom/9riSiaPtivE5ncl7cyW5hFqf2MzRd dF6q937ocgBd8d2VuOQleQ9N5gue1+du77soxfp8oY7dXNdwk5ZrngeYxz1umsjQ lBAI/3JYkKVhU5Rte3deJcMHe6xA+neIzvcfWbOZYrkdwWE+jLSKnDkDkCOXJnuP vOI8CAteaFctV2mfgXX2CmNDWVeFsyCwoQwbnCmuE/uiM5y235PcTrsyr6U+Zzs= =oQPK -END PGP SIGNATURE- 0x1B438BF4.asc Description: application/pgp-keys -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 8:35 PM, Justus Ranvier justusranv...@gmail.comwrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 06:19 PM, Wladimir wrote: If no one wants to volunteer resources to support the network anymore, we'll have failed. If the security of the network depends on a broken incentive model, then fix the design of the network so that economics works for you instead of against you. My solution would be quick to implement and could help increase the number of nodes in the short term. It will also smooth the way towards splitting off the wallet from Bitcoin Core, by giving people a choice what wallet they use with their full node. That doesn't preclude looking for longer-term solutions that change the incentive structure, but that is much more difficult and risky. I don't believe it's a matter of 'fixing that instead' in a few hours, though I nevertheless look forward to your pull request. Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 11:35 AM, Justus Ranvier justusranv...@gmail.com wrote: If the security of the network depends on a broken incentive model, then fix the design of the network so that economics works for you instead of against you. Who says anything about a broken incentive model. You've made past claims about resource requirements that I think made no sense and then failed to defend them when they were challenge. With suitable software improvements running a full node could be done in as little as a few gigabytes in disk space (e.g. cost 25-50 cents), and as 50-100kbit/sec bandwidth in and out ongoing, and a moderate amount of ram. Power costs are already just a few cents per month. By far the greatest cost is the figuring out and setting up part, which bundling could fix. The exact resources could be tunable to what the users are willing and able to contribute. If improved marginal security and privacy in addition to supporting the network is not enough incentive to overcome costs like these then Bitcoin is already doomed. I think that fundamental costs aren't an issue at all, just implementation warts and education are. Part of asking the question is feeling out which improvements need to happen first, and what the prospects of getting the bundling going once those improvements are made. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 06:50 PM, Gregory Maxwell wrote: Who says anything about a broken incentive model. You've made past claims about resource requirements that I think made no sense and then failed to defend them when they were challenge. Anyone reading the archives of the list will see about triple the number of people independently confirming the resource usage problem than they will see denying it, so I'm not particularly worried. - -- Support online privacy by using email encryption whenever possible. Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTRZhsAAoJECoisBQbQ4v0iD8H/A1RZyLJN6f+zt5x78CFgRdp zqZua3NBwUN2L9mI/PU1NADtxoXgkq49XusHRc+bjLu17VxGMUOsmB+AeK3VU4sN BVC1qIcWCa+OPMkgnMFrdydz4OGxX5xiJoCNsqveZIEYc8nQoCDfsn/fwqL50/Ct NfmPzh/cW7uUZ+h2bBgduGD3fzpQBlnnswAbHHX3FbBh9MvcaCfROeXXqUCedWj9 H2qVCNWTkIFJksJkqyF3f+KfedOTIyUwflx/0niBvzXcP9w4oK+WpBApHACWxiSA H3KHYNF0s2/fs+mIkabPLRX1PIUk15ln29FapoDvlHW9jzDE92x6JFnMppI4rEs= =ILks -END PGP SIGNATURE- 0x1B438BF4.asc Description: application/pgp-keys -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Jenkins pull-tester
I would like to set up my own bitcoin pull-tester on Jenkins. Are there any instructions or guidance written anywhere? Drak -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 6:09 PM, Thomas Voegtlin thoma...@gmx.de wrote: Le 09/04/2014 17:54, Gregory Maxwell a écrit : Sadly today Electrum requires more than a full node, it requires a number of large additional indexes over what a full node has and pruning is precluded. I don't think that increasing the resource utilization of the node is a good way to go there for the purposes expressed here. (not that electrum couldn't be used here, but not unmodified without the resource usage increasing route) Electrum uses two large indexes: address - utxo (patricia tree, aka ultimate blockchain compression, see thread started by Alan Reiner in the bitcointalk forum) Thanks for the explanation. Adding a RPC call for a address - utxo query wouldn't be a big deal. It has been requested before for other purposes as well, all the better if it helps for interaction with Electrum. Spent history would be involve a much larger index, and it's not likely that will end up in bitcoin Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
I believe there're plenty bitcoind instances running, but they don't have configured port forwarding properly.There's uPNP support in bitcoind, but it works only on simple setups. Maybe there're some not yet considered way how to expose these *existing* instances to Internet, to strenghten the network. Maybe just self-test indicating the node is not reachable from outside (together with short howto like in some torrent clients). These days IPv6 is slowly deploying to server environments, but maybe there's some simple way how to bundle ipv6 tunnelling into bitcoind so any instance will become ipv6-reachable automatically? Maybe there're other ideas how to improve current situation without needs of reworking the architecture. Marek On Wed, Apr 9, 2014 at 9:33 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier justusranv...@gmail.com wrote: Anyone reading the archives of the list will see about triple the number of people independently confirming the resource usage problem than they will see denying it, so I'm not particularly worried. The list has open membership, there is no particular qualification or background required to post here. Optimal use of an information source requires critical reading and understanding the limitations of the medium. Counting comments is usually not a great way to assess technical considerations on an open public forum. Doubly so because those comments were not actually talking about the same thing I am talking about. Existing implementations are inefficient in many known ways (and, no doubt, some unknown ones). This list is about developing protocol and implementations including improving their efficiency. When talking about incentives the costs you need to consider are the costs of the best realistic option. As far as I know there is no doubt from anyone technically experienced that under the current network rules full nodes can be operated with vastly less resources than current implementations use, it's just a question of the relatively modest implementation improvements. When you argue that Bitcoin doesn't have the right incentives (and thus something??) I retort that the actual resource _requirements_ are for the protocol very low. I gave specific example numbers to enable correction or clarification if I've said something wrong or controversial. Pointing out that existing implementations are not that currently as efficient as the underlying requirements and that some large number of users do not like the efficiency of existing implementations doesn't tell me anything I disagree with or didn't already know. Whats being discussed around here contributes to prioritizing improvements over the existing implementations. I hope this clarifies something. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
Another idea: Integrate torrent download of bootstrap.dat into bitcoind. Normal user (especially a beginner) won't learn how to download bootstrap separately and import it into bitcoind; he simply give up the synchronization once he realize it takes too much time. From my experience downloading the bootstrap significantly improves catching the blockchain, which may attract some more users to run bitcoind. Not sure about C++, but simple torrent client in python is like 30 lines of code (using libtorrent). Marek On Wed, Apr 9, 2014 at 10:12 PM, slush sl...@centrum.cz wrote: I believe there're plenty bitcoind instances running, but they don't have configured port forwarding properly.There's uPNP support in bitcoind, but it works only on simple setups. Maybe there're some not yet considered way how to expose these *existing* instances to Internet, to strenghten the network. Maybe just self-test indicating the node is not reachable from outside (together with short howto like in some torrent clients). These days IPv6 is slowly deploying to server environments, but maybe there's some simple way how to bundle ipv6 tunnelling into bitcoind so any instance will become ipv6-reachable automatically? Maybe there're other ideas how to improve current situation without needs of reworking the architecture. Marek On Wed, Apr 9, 2014 at 9:33 PM, Gregory Maxwell gmaxw...@gmail.comwrote: On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier justusranv...@gmail.com wrote: Anyone reading the archives of the list will see about triple the number of people independently confirming the resource usage problem than they will see denying it, so I'm not particularly worried. The list has open membership, there is no particular qualification or background required to post here. Optimal use of an information source requires critical reading and understanding the limitations of the medium. Counting comments is usually not a great way to assess technical considerations on an open public forum. Doubly so because those comments were not actually talking about the same thing I am talking about. Existing implementations are inefficient in many known ways (and, no doubt, some unknown ones). This list is about developing protocol and implementations including improving their efficiency. When talking about incentives the costs you need to consider are the costs of the best realistic option. As far as I know there is no doubt from anyone technically experienced that under the current network rules full nodes can be operated with vastly less resources than current implementations use, it's just a question of the relatively modest implementation improvements. When you argue that Bitcoin doesn't have the right incentives (and thus something??) I retort that the actual resource _requirements_ are for the protocol very low. I gave specific example numbers to enable correction or clarification if I've said something wrong or controversial. Pointing out that existing implementations are not that currently as efficient as the underlying requirements and that some large number of users do not like the efficiency of existing implementations doesn't tell me anything I disagree with or didn't already know. Whats being discussed around here contributes to prioritizing improvements over the existing implementations. I hope this clarifies something. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 10:12 PM, slush sl...@centrum.cz wrote: Maybe there're other ideas how to improve current situation without needs of reworking the architecture. Nothing I've proposed here would require larger changes to the architecture then were already planned. After SPV lands we are going to split off the wallet, and that will need an interface to an bitcoind to allow 'running with full node'. If that can be generalized to be useful for other (SPV) clients as well, that would be useful, hence I asked for input. It of course doesn't preclude also looking for other solutions. Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 10:31 PM, slush sl...@centrum.cz wrote: Another idea: Integrate torrent download of bootstrap.dat into bitcoind. Normal user (especially a beginner) won't learn how to download bootstrap separately and import it into bitcoind; he simply give up the synchronization once he realize it takes too much time. From my experience downloading the bootstrap significantly improves catching the blockchain, which may attract some more users to run bitcoind. Not sure about C++, but simple torrent client in python is like 30 lines of code (using libtorrent). Parallel block download would be better for that. No need to involve bittorrent. But please let's not derail this thread, this is not about other solutions for faster block download or such, let's keep it focused. Wladimir -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Apr 9, 2014, at 8:12 PM, slush sl...@centrum.cz wrote: These days IPv6 is slowly deploying to server environments, but maybe there's some simple way how to bundle ipv6 tunnelling into bitcoind so any instance will become ipv6-reachable automatically? Teredo is available by default on Microsoft systems and it's actually very common to see Teredo addresses as peers in torrents - this should work for bitcoin too, though I'm not sure if an app needs to set special flags to gain access to it.. there are probably some security settings around it. In the US, ATT/CenturyLink provide IPv6 by way of 6RD for DSL customers, Comcast has native IPv6 on residential (but not business) cable modems, and of course those who want to can always set up with a tunnel broker like Hurricane Electric - they even let you use your own IPv6 addresses. IPv6 is great, but having an application running its own tunnels would not be a good way to leverage it. Probably what's keeping a lot of them from being reachable is that most people just plug their CPE into a NAT router (without IPv6). Teredo can help here though. Putting an IPv6 checkbox in the settings with a link to an explanation might help with education and result in smarter operators who can make their nodes reachable. A built in checker would be even better - something like a checklist showing red/green for different aspects of reachability (v4/teredo/v6). -Laszlo -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
I've advocated for this in the past, and reasonable counter-arguments I was presented with are: (1) bittorrent is horribly insecure - it would be easy to DoS the initial block download if that were the goal, and (2) there's a reasonable pathway to doing this all in-protocol, so there's no reason to introduce external dependencies. On 04/09/2014 01:31 PM, slush wrote: Another idea: Integrate torrent download of bootstrap.dat into bitcoind. Normal user (especially a beginner) won't learn how to download bootstrap separately and import it into bitcoind; he simply give up the synchronization once he realize it takes too much time. From my experience downloading the bootstrap significantly improves catching the blockchain, which may attract some more users to run bitcoind. Not sure about C++, but simple torrent client in python is like 30 lines of code (using libtorrent). Marek On Wed, Apr 9, 2014 at 10:12 PM, slush sl...@centrum.cz mailto:sl...@centrum.cz wrote: I believe there're plenty bitcoind instances running, but they don't have configured port forwarding properly.There's uPNP support in bitcoind, but it works only on simple setups. Maybe there're some not yet considered way how to expose these *existing* instances to Internet, to strenghten the network. Maybe just self-test indicating the node is not reachable from outside (together with short howto like in some torrent clients). These days IPv6 is slowly deploying to server environments, but maybe there's some simple way how to bundle ipv6 tunnelling into bitcoind so any instance will become ipv6-reachable automatically? Maybe there're other ideas how to improve current situation without needs of reworking the architecture. Marek On Wed, Apr 9, 2014 at 9:33 PM, Gregory Maxwell gmaxw...@gmail.com mailto:gmaxw...@gmail.com wrote: On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier justusranv...@gmail.com mailto:justusranv...@gmail.com wrote: Anyone reading the archives of the list will see about triple the number of people independently confirming the resource usage problem than they will see denying it, so I'm not particularly worried. The list has open membership, there is no particular qualification or background required to post here. Optimal use of an information source requires critical reading and understanding the limitations of the medium. Counting comments is usually not a great way to assess technical considerations on an open public forum. Doubly so because those comments were not actually talking about the same thing I am talking about. Existing implementations are inefficient in many known ways (and, no doubt, some unknown ones). This list is about developing protocol and implementations including improving their efficiency. When talking about incentives the costs you need to consider are the costs of the best realistic option. As far as I know there is no doubt from anyone technically experienced that under the current network rules full nodes can be operated with vastly less resources than current implementations use, it's just a question of the relatively modest implementation improvements. When you argue that Bitcoin doesn't have the right incentives (and thus something??) I retort that the actual resource _requirements_ are for the protocol very low. I gave specific example numbers to enable correction or clarification if I've said something wrong or controversial. Pointing out that existing implementations are not that currently as efficient as the underlying requirements and that some large number of users do not like the efficiency of existing implementations doesn't tell me anything I disagree with or didn't already know. Whats being discussed around here contributes to prioritizing improvements over the existing implementations. I hope this clarifies something. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net mailto:Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets
On Wed, Apr 9, 2014 at 1:36 PM, Mark Friedenbach m...@monetize.io wrote: (2) there's a reasonable pathway to doing this all in-protocol, so there's no reason to introduce external dependencies. Not just a speculative pathway. Pieter implemented headers first: https://github.com/sipa/bitcoin/tree/headersfirst and it was everything we hoped it would be— it easily can saturate residential broadband, produces less load hot-spotting, copes with unreliable peers, etc. It's now just slowly making its way into Bitcoin core. -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
