Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-05-04 Thread Tier Nolan
On Fri, Apr 11, 2014 at 5:54 PM, Gregory Maxwell wrote: > For the non-error-coded case I believe nodes > with random spans of blocks works out asymptotically to the same > failure rates as random. > If each "block" is really 512 blocks in sequence, then each "slot" is more likely to be hit. It

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-11 Thread Gregory Maxwell
On Thu, Apr 10, 2014 at 10:30 AM, Tier Nolan wrote: > Error correction is an interesting suggestion. Though I mentioned it, it was in jest— I think right now it would be an over-design at least for the basic protocol. Also, storing 'random' blocks has some locality problems, when verifying block

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tier Nolan
Error correction is an interesting suggestion. If there was 1 nodes and each stored 0.1% of the blocks, at random, then the odds of a block not being stored is 45 in a million. Blocks are stored on average 10 times, so there is already reasonable redundancy. With 1 million blocks, 45 would b

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 07:50:55 GMT-04:00, Gregory Maxwell wrote: >(Just be glad I'm not suggesting coding the entire blockchain with an >error correcting code so that it doesn't matter which subset you're >holding) I forgot to ask last night: if you d

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 07:45:16 GMT-04:00, Mike Hearn wrote: >> >> Oh yeah, credit goes to Mike Hearn for the payment channels, and if >I'm >> correct, for the hub concept as well. >> > >Actually, the design is from Satoshi and Matt did most of the >imp

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Gregory Maxwell
On Thu, Apr 10, 2014 at 4:32 AM, Pieter Wuille wrote: > There were earlier discussions. On this list. > The two ideas were either using one or a few service bits to indicate > availability of blocks, or to extend addr messages with some flags to > indicate this information. > > I wonder whether

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
> > Oh yeah, credit goes to Mike Hearn for the payment channels, and if I'm > correct, for the hub concept as well. > Actually, the design is from Satoshi and Matt did most of the implementation work last year during a Google internship. Though I ended up doing a lot of work on it too. We actually

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 07:32:44 GMT-04:00, Pieter Wuille wrote: >There were earlier discussions. > >The two ideas were either using one or a few service bits to indicate >availability of blocks, or to extend addr messages with some flags to >indicate t

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 06:44:32 GMT-04:00, Tamas Blummer wrote: >Thanks, Peter and you convinced me. I run away with a thought. > >It’d be great to find a spot to deploy payment channels, but I agree >this is not it. No problem! I'm sure we'll see pa

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Pieter Wuille
There were earlier discussions. The two ideas were either using one or a few service bits to indicate availability of blocks, or to extend addr messages with some flags to indicate this information. I wonder whether we can't have a hybrid: bits to indicate general degree of availability of blocks

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
> > What would this involve? > > Do you know of any previous work towards this? > Chain pruning is a fairly complicated project, partly because it spans codebases. For instance if you try and implement it *just* by changing Bitcoin Core, you will break all the SPV clients based on bitcoinj (i.e. a

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Wladimir
On Thu, Apr 10, 2014 at 8:04 AM, Tamas Blummer wrote: > Serving headers should be default but storing and serving full blocks > configurable to ranges, so people can tailor to their bandwith and space > available. > I do agree that it is important. This does require changes to the P2P protocol,

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
Thanks, Peter and you convinced me. I run away with a thought. It’d be great to find a spot to deploy payment channels, but I agree this is not it. Tamas Blummer http://bitsofproof.com On 10.04.2014, at 12:40, Mike Hearn wrote: > 1) There is no catch 22 as there are plenty of ways getting bit

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
> > 1) There is no catch 22 as there are plenty of ways getting bitcoin > without bootstrapping a full node. > I think I maybe wasn't clear. To spend coins you need transaction data. Today, the dominant model is that people get that data by scanning the block chain. If you can obtain the transacti

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10 April 2014 05:17:28 GMT-04:00, Mike Hearn wrote: >> >> I find it is odd that we who hold the key to instant machine to >machine >> micro payments do not use it to incentivise committing resources to >the >> network. >> > >It's not a new idea

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
I know the idea is not new. Just bringing it up to emphasize that if we don’t use it how could we expect other networks using it. Machine to machine micro payments could become the killer application for Bitcoin. 1) There is no catch 22 as there are plenty of ways getting bitcoin without bootst

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 But we >have >to be realistic. Desktop tower machines that are always on are dying >and >will not be coming back. Not a single person I know uses them anymore, >they >have been wiped out in favour of laptops. This is why, given the tiny >size >of

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
> > I find it is odd that we who hold the key to instant machine to machine > micro payments do not use it to incentivise committing resources to the > network. > It's not a new idea, obviously, but there are some practical consequences: 1) To pay a node for serving, you have to have bitcoins. To

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
You ask why people would install this ? I find it is odd that we who hold the key to instant machine to machine micro payments do not use it to incentivise committing resources to the network. What about serving archive blocks to peers paying for it ? Tamas Blummer http://bitsofproof.com On 10.

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
It's an optimisation problem. Home environments are much more hostile than servers are due to things like virus scanners, wildly varying memory pressure as apps are started and shut down, highly asymmetrical upstream versus downstream bandwidth, complicated nat setups, people who only use laptops

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Thu, Apr 10, 2014 at 8:38 AM, Mike Hearn wrote: > I tend to agree with slush here - counting the IPs in addr broadcasts > often gives a number like 100,000 vs just 10,000 for actually reachable > nodes (or less). It seems like optimising the NAT tunneling code would > help. Starting by adding

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mike Hearn
I tend to agree with slush here - counting the IPs in addr broadcasts often gives a number like 100,000 vs just 10,000 for actually reachable nodes (or less). It seems like optimising the NAT tunneling code would help. Starting by adding more diagnostic stuff to the GUI. STUN support may also help.

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
Hi Wladimir, If the motivation of the SPV wallet is to radically extend functionality, as in my case, then the index is specific to the added features and the subset of the blockchain that is of interest for the wallet. As you also point out, adding huge generic purpose indices to core would rat

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 1:36 PM, Mark Friedenbach wrote: > (2) there's a reasonable pathway to doing this all in-protocol, so there's > no reason to introduce external dependencies. Not just a speculative pathway. Pieter implemented headers first: https://github.com/sipa/bitcoin/tree/headersfirst

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mark Friedenbach
I've advocated for this in the past, and reasonable counter-arguments I was presented with are: (1) bittorrent is horribly insecure - it would be easy to DoS the initial block download if that were the goal, and (2) there's a reasonable pathway to doing this all in-protocol, so there's no reason to

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Laszlo Hanyecz
On Apr 9, 2014, at 8:12 PM, slush wrote: > > These days IPv6 is slowly deploying to server environments, but maybe there's > some simple way how to bundle ipv6 tunnelling into bitcoind so any instance > will become ipv6-reachable automatically? > Teredo is available by default on Microsoft

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread slush
We definitely *need* lightweight bitcoin router / "core" which can be easily deployed anywhere. No disagreement here. I just wanted to remind that there're actually much more running nodes *already* and maybe converting those hidden nodes to publicly reachable nodes may be way easier than attracti

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 10:31 PM, slush wrote: > Another idea: Integrate torrent download of bootstrap.dat into bitcoind. > Normal user (especially a beginner) won't learn how to download bootstrap > separately and import it into bitcoind; he simply give up the > synchronization once he realize it

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 10:12 PM, slush wrote: > Maybe there're other ideas how to improve current situation without needs > of reworking the architecture. > Nothing I've proposed here would require larger changes to the architecture then were already planned. After SPV lands we are going to spli

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread slush
Another idea: Integrate torrent download of bootstrap.dat into bitcoind. Normal user (especially a beginner) won't learn how to download bootstrap separately and import it into bitcoind; he simply give up the synchronization once he realize it takes too much time. From my experience downloading the

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread slush
I believe there're plenty bitcoind instances running, but they don't have configured port forwarding properly.There's uPNP support in bitcoind, but it works only on simple setups. Maybe there're some not yet considered way how to expose these *existing* instances to Internet, to strenghten the net

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier wrote: > Anyone reading the archives of the list will see about triple the > number of people independently confirming the resource usage problem > than they will see denying it, so I'm not particularly worried. The list has open membership, there i

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 6:09 PM, Thomas Voegtlin wrote: > Le 09/04/2014 17:54, Gregory Maxwell a écrit : > > > Sadly today Electrum requires more than a full node, it requires a > > number of large additional indexes over what a full node has and > > pruning is precluded. I don't think that increa

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Justus Ranvier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 06:50 PM, Gregory Maxwell wrote: > Who says anything about a broken incentive model. You've made past > claims about resource requirements that I think made no sense and > then failed to defend them when they were challenge. Anyone read

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 11:35 AM, Justus Ranvier wrote: > If the security of the network depends on a broken incentive model, > then fix the design of the network so that economics works for you > instead of against you. Who says anything about a broken incentive model. You've made past claims abo

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 8:35 PM, Justus Ranvier wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 04/09/2014 06:19 PM, Wladimir wrote: > > If no one wants to volunteer resources to support the network > > anymore, we'll have failed. > > If the security of the network depends on a brok

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Justus Ranvier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 06:19 PM, Wladimir wrote: > If no one wants to volunteer resources to support the network > anymore, we'll have failed. If the security of the network depends on a broken incentive model, then fix the design of the network so that econom

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Kevin
On 4/9/2014 11:29 AM, Wladimir wrote: Hello, This is primarily aimed at developers of SPV wallets. The recently reported decrease in number of full nodes could have several reasons, one of them that less people are running Bitcoin Core for the wallet because the other wallets are getting ahea

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 8:00 PM, Mike Hearn wrote: > The right way to start with this, if anyone cares, is to add > instrumentation to existing SPV wallet apps to report back to home base how > long they are running for, how much disk space / RAM they have, and > possibly what kind of hardware. >

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 9 April 2014 13:50:03 GMT-04:00, Tamas Blummer wrote: >Block header has to be available in SPV and also in an UTXO only >storing core node, so why not serve it if bandwith allows. > >Serving any additional information like known peer adresses o

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mike Hearn
The right way to start with this, if anyone cares, is to add instrumentation to existing SPV wallet apps to report back to home base how long they are running for, how much disk space / RAM they have, and possibly what kind of hardware. I *strongly* suspect that the vast majority of SPV wallets ar

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
Block header has to be available in SPV and also in an UTXO only storing core node, so why not serve it if bandwith allows. Serving any additional information like known peer adresses or known full blocks is certainly beneficial and should be offered if at hand. Regards, Tamas Blummer http://b

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 9 April 2014 12:27:13 GMT-04:00, Tamas Blummer wrote: >A border router that is not able to serve blocks is still protecting >consensus rules, that SPVs do not. >If the network would only consist of SPV nodes only then e.g. a >majority coalition

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 9 April 2014 13:33:19 GMT-04:00, Alex Mizrahi wrote: >> >> 1) It's more private. Bloom filters gives away quite accurate >statistical >> information about what coins you own to whom ever you happen to be >> connected too. An attacker can easily

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 7:33 PM, Alex Mizrahi wrote: > 1) It's more private. Bloom filters gives away quite accurate statistical >> information about what coins you own to whom ever you happen to be >> connected too. An attacker can easily use this to deanonymize you even if >> you don't reuse add

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Alex Mizrahi
> > 1) It's more private. Bloom filters gives away quite accurate statistical > information about what coins you own to whom ever you happen to be > connected too. An attacker can easily use this to deanonymize you even if > you don't reuse addresses; Tor does not help much against this attack. >

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 5:41 PM, Natanael wrote: > This could probably be done fairly easily by bundling Stratum (it's > not just for pools!) and allowing SPV wallets to ask Bitcoind to start > it (if you don't use it, there's no need to waste the resources), and > then connect to it. The point of

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
A border router that is not able to serve blocks is still protecting consensus rules, that SPVs do not. If the network would only consist of SPV nodes only then e.g. a majority coalition of miner could increase their reward at will. Archives need a different solution. Regards, Tamas Blummer ht

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Two talking points for said developers yo their user re: "Why use a full node?": 1) It's more private. Bloom filters gives away quite accurate statistical information about what coins you own to whom ever you happen to be connected too. An attacke

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mark Friedenbach
On 04/09/2014 09:09 AM, Tamas Blummer wrote: > Yes, SPV is a sufficient API to a trusted node to build sophisticated > features not offered by the core. > SPV clients of the border router will build their own archive and > indices based on their interest of the chain therefore the > border router c

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
I am glad that SPV wallets are discussed outside the scope of mobile devices! Yes, SPV is a sufficient API to a trusted node to build sophisticated features not offered by the core. SPV clients of the border router will build their own archive and indices based on their interest of the chain the

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Thomas Voegtlin
Le 09/04/2014 17:54, Gregory Maxwell a écrit : > Sadly today Electrum requires more than a full node, it requires a > number of large additional indexes over what a full node has and > pruning is precluded. I don't think that increasing the resource > utilization of the node is a good way to go th

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 8:41 AM, Natanael wrote: > This could probably be done fairly easily by bundling Stratum (it's > not just for pools!) and allowing SPV wallets to ask Bitcoind to start > it (if you don't use it, there's no need to waste the resources), and > then connect to it. The point of

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 8:42 AM, Brian Hoffman wrote: > How would this affect the user in terms of disk storage? They're going to > get hammered on space constraints aren't they? If it's not required how > likely are users to enable this? If Bitcoin core activates pruning a full node can be suppor

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Natanael
This could probably be done fairly easily by bundling Stratum (it's not just for pools!) and allowing SPV wallets to ask Bitcoind to start it (if you don't use it, there's no need to waste the resources), and then connect to it. The point of using Stratum is that it already is being used by Electru

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Brian Hoffman
How would this affect the user in terms of disk storage? They're going to get hammered on space constraints aren't they? If it's not required how likely are users to enable this? On Wed, Apr 9, 2014 at 11:29 AM, Wladimir wrote: > Hello, > > This is primarily aimed at developers of SPV wallets.

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
YES Such a bitcoind is what I called border router in a previous mail. Yes, SPV wallets are getting ahead of features, so people will use them also because on size just does not fit all, but all want to ensure being on the same trunk of the chain. Therefore serious user of Bitcoin run a bitcoi