Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-05-04 Thread Tier Nolan
On Fri, Apr 11, 2014 at 5:54 PM, Gregory Maxwell gmaxw...@gmail.com wrote: For the non-error-coded case I believe nodes with random spans of blocks works out asymptotically to the same failure rates as random. If each block is really 512 blocks in sequence, then each slot is more likely to

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-11 Thread Gregory Maxwell
On Thu, Apr 10, 2014 at 10:30 AM, Tier Nolan tier.no...@gmail.com wrote: Error correction is an interesting suggestion. Though I mentioned it, it was in jest— I think right now it would be an over-design at least for the basic protocol. Also, storing 'random' blocks has some locality problems,

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
Hi Wladimir, If the motivation of the SPV wallet is to radically extend functionality, as in my case, then the index is specific to the added features and the subset of the blockchain that is of interest for the wallet. As you also point out, adding huge generic purpose indices to core would

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
I tend to agree with slush here - counting the IPs in addr broadcasts often gives a number like 100,000 vs just 10,000 for actually reachable nodes (or less). It seems like optimising the NAT tunneling code would help. Starting by adding more diagnostic stuff to the GUI. STUN support may also

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Wladimir
On Thu, Apr 10, 2014 at 8:38 AM, Mike Hearn m...@plan99.net wrote: I tend to agree with slush here - counting the IPs in addr broadcasts often gives a number like 100,000 vs just 10,000 for actually reachable nodes (or less). It seems like optimising the NAT tunneling code would help.

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
It's an optimisation problem. Home environments are much more hostile than servers are due to things like virus scanners, wildly varying memory pressure as apps are started and shut down, highly asymmetrical upstream versus downstream bandwidth, complicated nat setups, people who only use laptops

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
You ask why people would install this ? I find it is odd that we who hold the key to instant machine to machine micro payments do not use it to incentivise committing resources to the network. What about serving archive blocks to peers paying for it ? Tamas Blummer http://bitsofproof.com On

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
I find it is odd that we who hold the key to instant machine to machine micro payments do not use it to incentivise committing resources to the network. It's not a new idea, obviously, but there are some practical consequences: 1) To pay a node for serving, you have to have bitcoins. To get

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 But we have to be realistic. Desktop tower machines that are always on are dying and will not be coming back. Not a single person I know uses them anymore, they have been wiped out in favour of laptops. This is why, given the tiny size of the

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
I know the idea is not new. Just bringing it up to emphasize that if we don’t use it how could we expect other networks using it. Machine to machine micro payments could become the killer application for Bitcoin. 1) There is no catch 22 as there are plenty of ways getting bitcoin without

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 05:17:28 GMT-04:00, Mike Hearn m...@plan99.net wrote: I find it is odd that we who hold the key to instant machine to machine micro payments do not use it to incentivise committing resources to the network. It's not a new

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
1) There is no catch 22 as there are plenty of ways getting bitcoin without bootstrapping a full node. I think I maybe wasn't clear. To spend coins you need transaction data. Today, the dominant model is that people get that data by scanning the block chain. If you can obtain the transaction

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
Thanks, Peter and you convinced me. I run away with a thought. It’d be great to find a spot to deploy payment channels, but I agree this is not it. Tamas Blummer http://bitsofproof.com On 10.04.2014, at 12:40, Mike Hearn m...@plan99.net wrote: 1) There is no catch 22 as there are plenty of

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Wladimir
On Thu, Apr 10, 2014 at 8:04 AM, Tamas Blummer ta...@bitsofproof.comwrote: Serving headers should be default but storing and serving full blocks configurable to ranges, so people can tailor to their bandwith and space available. I do agree that it is important. This does require changes to

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
What would this involve? Do you know of any previous work towards this? Chain pruning is a fairly complicated project, partly because it spans codebases. For instance if you try and implement it *just* by changing Bitcoin Core, you will break all the SPV clients based on bitcoinj (i.e. all

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Pieter Wuille
There were earlier discussions. The two ideas were either using one or a few service bits to indicate availability of blocks, or to extend addr messages with some flags to indicate this information. I wonder whether we can't have a hybrid: bits to indicate general degree of availability of

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 06:44:32 GMT-04:00, Tamas Blummer ta...@bitsofproof.com wrote: Thanks, Peter and you convinced me. I run away with a thought. It’d be great to find a spot to deploy payment channels, but I agree this is not it. No problem! I'm

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 07:32:44 GMT-04:00, Pieter Wuille pieter.wui...@gmail.com wrote: There were earlier discussions. The two ideas were either using one or a few service bits to indicate availability of blocks, or to extend addr messages with some

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
Oh yeah, credit goes to Mike Hearn for the payment channels, and if I'm correct, for the hub concept as well. Actually, the design is from Satoshi and Matt did most of the implementation work last year during a Google internship. Though I ended up doing a lot of work on it too. We actually

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Gregory Maxwell
On Thu, Apr 10, 2014 at 4:32 AM, Pieter Wuille pieter.wui...@gmail.com wrote: There were earlier discussions. On this list. The two ideas were either using one or a few service bits to indicate availability of blocks, or to extend addr messages with some flags to indicate this information.

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 07:45:16 GMT-04:00, Mike Hearn m...@plan99.net wrote: Oh yeah, credit goes to Mike Hearn for the payment channels, and if I'm correct, for the hub concept as well. Actually, the design is from Satoshi and Matt did most of the

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 07:50:55 GMT-04:00, Gregory Maxwell gmaxw...@gmail.com wrote: (Just be glad I'm not suggesting coding the entire blockchain with an error correcting code so that it doesn't matter which subset you're holding) I forgot to ask last

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tier Nolan
Error correction is an interesting suggestion. If there was 1 nodes and each stored 0.1% of the blocks, at random, then the odds of a block not being stored is 45 in a million. Blocks are stored on average 10 times, so there is already reasonable redundancy. With 1 million blocks, 45 would

[Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
Hello, This is primarily aimed at developers of SPV wallets. The recently reported decrease in number of full nodes could have several reasons, one of them that less people are running Bitcoin Core for the wallet because the other wallets are getting ahead in both features and useability. It's

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
YES Such a bitcoind is what I called border router in a previous mail. Yes, SPV wallets are getting ahead of features, so people will use them also because on size just does not fit all, but all want to ensure being on the same trunk of the chain. Therefore serious user of Bitcoin run a

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Brian Hoffman
How would this affect the user in terms of disk storage? They're going to get hammered on space constraints aren't they? If it's not required how likely are users to enable this? On Wed, Apr 9, 2014 at 11:29 AM, Wladimir laa...@gmail.com wrote: Hello, This is primarily aimed at developers of

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 8:42 AM, Brian Hoffman brianchoff...@gmail.com wrote: How would this affect the user in terms of disk storage? They're going to get hammered on space constraints aren't they? If it's not required how likely are users to enable this? If Bitcoin core activates pruning a

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 8:41 AM, Natanael natanae...@gmail.com wrote: This could probably be done fairly easily by bundling Stratum (it's not just for pools!) and allowing SPV wallets to ask Bitcoind to start it (if you don't use it, there's no need to waste the resources), and then connect to

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Thomas Voegtlin
Le 09/04/2014 17:54, Gregory Maxwell a écrit : Sadly today Electrum requires more than a full node, it requires a number of large additional indexes over what a full node has and pruning is precluded. I don't think that increasing the resource utilization of the node is a good way to go there

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
I am glad that SPV wallets are discussed outside the scope of mobile devices! Yes, SPV is a sufficient API to a trusted node to build sophisticated features not offered by the core. SPV clients of the border router will build their own archive and indices based on their interest of the chain

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mark Friedenbach
On 04/09/2014 09:09 AM, Tamas Blummer wrote: Yes, SPV is a sufficient API to a trusted node to build sophisticated features not offered by the core. SPV clients of the border router will build their own archive and indices based on their interest of the chain therefore the border router core

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 5:41 PM, Natanael natanae...@gmail.com wrote: This could probably be done fairly easily by bundling Stratum (it's not just for pools!) and allowing SPV wallets to ask Bitcoind to start it (if you don't use it, there's no need to waste the resources), and then connect to

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Alex Mizrahi
1) It's more private. Bloom filters gives away quite accurate statistical information about what coins you own to whom ever you happen to be connected too. An attacker can easily use this to deanonymize you even if you don't reuse addresses; Tor does not help much against this attack. There

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 7:33 PM, Alex Mizrahi alex.mizr...@gmail.com wrote: 1) It's more private. Bloom filters gives away quite accurate statistical information about what coins you own to whom ever you happen to be connected too. An attacker can easily use this to deanonymize you even if you

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 9 April 2014 13:33:19 GMT-04:00, Alex Mizrahi alex.mizr...@gmail.com wrote: 1) It's more private. Bloom filters gives away quite accurate statistical information about what coins you own to whom ever you happen to be connected too. An

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 9 April 2014 12:27:13 GMT-04:00, Tamas Blummer ta...@bitsofproof.com wrote: A border router that is not able to serve blocks is still protecting consensus rules, that SPVs do not. If the network would only consist of SPV nodes only then e.g. a

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
Block header has to be available in SPV and also in an UTXO only storing core node, so why not serve it if bandwith allows. Serving any additional information like known peer adresses or known full blocks is certainly beneficial and should be offered if at hand. Regards, Tamas Blummer

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mike Hearn
The right way to start with this, if anyone cares, is to add instrumentation to existing SPV wallet apps to report back to home base how long they are running for, how much disk space / RAM they have, and possibly what kind of hardware. I *strongly* suspect that the vast majority of SPV wallets

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 9 April 2014 13:50:03 GMT-04:00, Tamas Blummer ta...@bitsofproof.com wrote: Block header has to be available in SPV and also in an UTXO only storing core node, so why not serve it if bandwith allows. Serving any additional information like

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 8:00 PM, Mike Hearn m...@plan99.net wrote: The right way to start with this, if anyone cares, is to add instrumentation to existing SPV wallet apps to report back to home base how long they are running for, how much disk space / RAM they have, and possibly what kind of

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Kevin
On 4/9/2014 11:29 AM, Wladimir wrote: Hello, This is primarily aimed at developers of SPV wallets. The recently reported decrease in number of full nodes could have several reasons, one of them that less people are running Bitcoin Core for the wallet because the other wallets are getting

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Justus Ranvier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 06:19 PM, Wladimir wrote: If no one wants to volunteer resources to support the network anymore, we'll have failed. If the security of the network depends on a broken incentive model, then fix the design of the network so that

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 8:35 PM, Justus Ranvier justusranv...@gmail.comwrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 06:19 PM, Wladimir wrote: If no one wants to volunteer resources to support the network anymore, we'll have failed. If the security of the network

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 11:35 AM, Justus Ranvier justusranv...@gmail.com wrote: If the security of the network depends on a broken incentive model, then fix the design of the network so that economics works for you instead of against you. Who says anything about a broken incentive model. You've

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Justus Ranvier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 06:50 PM, Gregory Maxwell wrote: Who says anything about a broken incentive model. You've made past claims about resource requirements that I think made no sense and then failed to defend them when they were challenge. Anyone

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 6:09 PM, Thomas Voegtlin thoma...@gmx.de wrote: Le 09/04/2014 17:54, Gregory Maxwell a écrit : Sadly today Electrum requires more than a full node, it requires a number of large additional indexes over what a full node has and pruning is precluded. I don't think

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier justusranv...@gmail.com wrote: Anyone reading the archives of the list will see about triple the number of people independently confirming the resource usage problem than they will see denying it, so I'm not particularly worried. The list has

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread slush
I believe there're plenty bitcoind instances running, but they don't have configured port forwarding properly.There's uPNP support in bitcoind, but it works only on simple setups. Maybe there're some not yet considered way how to expose these *existing* instances to Internet, to strenghten the

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread slush
Another idea: Integrate torrent download of bootstrap.dat into bitcoind. Normal user (especially a beginner) won't learn how to download bootstrap separately and import it into bitcoind; he simply give up the synchronization once he realize it takes too much time. From my experience downloading

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 10:12 PM, slush sl...@centrum.cz wrote: Maybe there're other ideas how to improve current situation without needs of reworking the architecture. Nothing I've proposed here would require larger changes to the architecture then were already planned. After SPV lands we are

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 10:31 PM, slush sl...@centrum.cz wrote: Another idea: Integrate torrent download of bootstrap.dat into bitcoind. Normal user (especially a beginner) won't learn how to download bootstrap separately and import it into bitcoind; he simply give up the synchronization once

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Laszlo Hanyecz
On Apr 9, 2014, at 8:12 PM, slush sl...@centrum.cz wrote: These days IPv6 is slowly deploying to server environments, but maybe there's some simple way how to bundle ipv6 tunnelling into bitcoind so any instance will become ipv6-reachable automatically? Teredo is available by default

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mark Friedenbach
I've advocated for this in the past, and reasonable counter-arguments I was presented with are: (1) bittorrent is horribly insecure - it would be easy to DoS the initial block download if that were the goal, and (2) there's a reasonable pathway to doing this all in-protocol, so there's no reason

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 1:36 PM, Mark Friedenbach m...@monetize.io wrote: (2) there's a reasonable pathway to doing this all in-protocol, so there's no reason to introduce external dependencies. Not just a speculative pathway. Pieter implemented headers first: