Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-16 Thread Ricardo Filipe
Of course. My proposal was just for the pruned nodes. I.e. You would have a majority (maybe not even a majority required) of nodes storing the whole blockchain and pruned nodes would store "random" parts of the blockchain, according to the resources they have, which would be organized as a DHT. 20

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-16 Thread Jeff Garzik
On Thu, May 16, 2013 at 7:26 AM, Ricardo Filipe wrote: > We would only end up with few copies of the historic data if users > could choose what parts of the blockchain to store. Simply store > chunks randomly, according to users available space, and give priority > to the "N most recent" chunks to

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-16 Thread Ricardo Filipe
We would only end up with few copies of the historic data if users could choose what parts of the blockchain to store. Simply store chunks randomly, according to users available space, and give priority to the "N most recent" chunks to have more replicas in the network. You don't need bittorrent s

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-06 Thread Pieter Wuille
On Mon, May 06, 2013 at 10:19:35AM +0200, Mike Hearn wrote: > You are welcome to optimise P2P addr broadcasts or develop better bootstrap > mechanisms. I think John's actually has a point here. If we're judging the quality of a protocol change by how compatible it is with DNS seeding, then we're c

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-06 Thread Mike Hearn
You are welcome to optimise P2P addr broadcasts or develop better bootstrap mechanisms. On Sun, May 5, 2013 at 3:12 PM, John Dillon wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Sorry I should have used the word bootstrapping there rather than > discovery. > But again I think th

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-05 Thread John Dillon
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Sorry I should have used the word bootstrapping there rather than discovery. But again I think that shows my point clearly. Centralized methods like DNS should be used for as little as possible, just simple initial bootstrapping, and focus the develo

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-04 Thread Jeff Garzik
On Sat, May 4, 2013 at 2:07 PM, John Dillon wrote: > After all Peter, just like you have implemented alternate block header > distribution over twitter, in the future we should have many different means > of > peer discovery. Right now we have DNS seeds, a fixed list, and IRC discovery > that doe

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-04 Thread John Dillon
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 I think you too should ask yourself why you are putting so much effort into optimizing a centralized service, the DNS seeds, rather than putting effort into optimizing the P2P peer discovery instead. DNS seeds are a necessary evil, one that shouldn't

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-03 Thread Peter Todd
On Fri, May 03, 2013 at 05:02:26PM +0200, Mike Hearn wrote: > > If you're going to take a step like that, the > > should be rounded off, perhaps to some number of bits, or you'll allow > > DNS caching to be defeated. > > > > Don't the seeds already set small times? I'm not sure we want these > re

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-03 Thread Mike Hearn
> If you're going to take a step like that, the > should be rounded off, perhaps to some number of bits, or you'll allow > DNS caching to be defeated. > Don't the seeds already set small times? I'm not sure we want these responses to be cacheable, otherwise there's a risk of a wall of traffic sud

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-03 Thread Peter Todd
On Fri, May 03, 2013 at 04:06:29PM +0200, Mike Hearn wrote: > That's true, but we can extend the DNS seeding protocol a little bit - you > could query .dnsseed.whatever.com and the DNS server > then only returns nodes it knows matches your requirement. If you're going to take a step like that, the

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-03 Thread Mike Hearn
> Yes, I like that better than broadcasting the exact height starting at > which you serve (though I would put that information immediately in the > version announcement). I don't think we can rely on the addr broadcasting > mechanism for fast information exchange anyway. One more problem with this

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-03 Thread Pieter Wuille
(generic comment on the discussion that spawned off: ideas about how to allow additional protocols for block exchange are certainly interesting, and in the long term we should certainly consider that. For now I'd like to keep this about the more immediate way forward with making the P2P protocol no

Re: [Bitcoin-development] Service bits for pruned nodes

2013-05-01 Thread Jeff Garzik
On Sun, Apr 28, 2013 at 11:51 AM, Pieter Wuille wrote: > Hello all, > > I think it is time to move forward with pruning nodes, i.e. nodes that fully > validate and relay blocks and transactions, but which do not keep (all) > historic blocks around, and thus cannot be queried for these. > > The big

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-30 Thread Brenton Camac
Sounds like this part of Bitcoin (block sharing) would definitely benefit from having a REST (HTTP) API. REST-based web APIs are a common feature of most online services these days. Makes writing other client services so much easier. Plus you get the benefit of the HTTP ecosystem for free (HT

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread Jay F
On 4/28/2013 8:55 PM, Peter Todd wrote: > On Mon, Apr 29, 2013 at 03:48:18AM +, John Dillon wrote: >> We can build this stuff incrementally I'll agree. It won't be the case that >> one >> in a thousand nodes serve up the part of the chain you need overnight. So >> many >> I am over engineerin

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread Peter Todd
On Mon, Apr 29, 2013 at 03:48:18AM +, John Dillon wrote: > We can build this stuff incrementally I'll agree. It won't be the case that > one > in a thousand nodes serve up the part of the chain you need overnight. So many > I am over engineering the solution with BitTorrent. I think that pret

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread John Dillon
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Mon, Apr 29, 2013 at 3:36 AM, Gregory Maxwell wrote: > On Sun, Apr 28, 2013 at 7:57 PM, John Dillon > wrote: >> Have we considered just leaving that problem to a different protocol such as >> BitTorrent? Offering up a few GB of storage capacity

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread Robert Backhaus
While I like the idea of a client using a DHT blockchain or UTXO list, I don't think that the reference client is the place for it. But it would make for a very interesting experimental project! On 29 April 2013 13:36, Gregory Maxwell wrote: > On Sun, Apr 28, 2013 at 7:57 PM, John Dillon > wro

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread Gregory Maxwell
On Sun, Apr 28, 2013 at 7:57 PM, John Dillon wrote: > Have we considered just leaving that problem to a different protocol such as > BitTorrent? Offering up a few GB of storage capacity is a nice idea but it > means we would soon have to add structure to the network to allow nodes to > find > eac

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread John Dillon
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 > But I also agree that its important that be splittable into > ranges > because otherwise when having to choose between serving historic data > and— say— 40 GB storage, a great many are going to choose not to serve > historic data... and so nodes

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread Gregory Maxwell
On Sun, Apr 28, 2013 at 9:29 AM, Mike Hearn wrote: > I'd imagined that nodes would be able to pick their own ranges to keep > rather than have fixed chosen intervals. "Everything or two weeks" is rather X most recent is special for two reasons: It meshes well with actual demand, and the data is

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread Mike Hearn
That's true. It can be perhaps be represented as "I keep the last N blocks" and then most likely for any given node the policy doesn't change all that fast, so if you know the best chain height you can calculate which nodes have what. > Disconnecting in case something is requested that isn't serv

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread Pieter Wuille
On Sun, Apr 28, 2013 at 6:29 PM, Mike Hearn wrote: > I'd imagined that nodes would be able to pick their own ranges to keep > rather than have fixed chosen intervals. "Everything or two weeks" is > rather restrictive - presumably node operators are constrained by physical > disk space, which mean

Re: [Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread Mike Hearn
I'd imagined that nodes would be able to pick their own ranges to keep rather than have fixed chosen intervals. "Everything or two weeks" is rather restrictive - presumably node operators are constrained by physical disk space, which means the quantity of blocks they would want to keep can vary wit

[Bitcoin-development] Service bits for pruned nodes

2013-04-28 Thread Pieter Wuille
Hello all, I think it is time to move forward with pruning nodes, i.e. nodes that fully validate and relay blocks and transactions, but which do not keep (all) historic blocks around, and thus cannot be queried for these. The biggest roadblock is making sure new and old nodes that start up are ab