Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-13 Thread Tier Nolan
On Wed, May 13, 2015 at 6:19 AM, Daniel Kraft d...@domob.eu wrote: 2) Divide the range of all blocks into intervals with exponentially growing size. I. e., something like this: 1, 1, 2, 2, 4, 4, 8, 8, 16, 16, ... Interesting. This can be combined with the system I suggested. A node

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread gabe appleton
Yes, but that just increases the incentive for partially-full nodes. It would add to the assumed-small number of full nodes. Or am I misunderstanding? On Tue, May 12, 2015 at 12:05 PM, Jeff Garzik jgar...@bitpay.com wrote: A general assumption is that you will have a few archive nodes with the

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread Peter Todd
On Tue, May 12, 2015 at 09:05:44AM -0700, Jeff Garzik wrote: A general assumption is that you will have a few archive nodes with the full blockchain, and a majority of nodes are pruned, able to serve only the tail of the chains. Hmm? Lots of people are tossing around ideas for partial

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread Jeff Garzik
A general assumption is that you will have a few archive nodes with the full blockchain, and a majority of nodes are pruned, able to serve only the tail of the chains. On Tue, May 12, 2015 at 8:26 AM, gabe appleton gapplet...@gmail.com wrote: Hi, There's been a lot of talk in the rest of the

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread Jeff Garzik
True. Part of the issue rests on the block sync horizon/cliff. There is a value X which is the average number of blocks the 90th percentile of nodes need in order to sync. It is sufficient for the [semi-]pruned nodes to keep X blocks, after which nodes must fall back to archive nodes for older

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread gabe appleton
Yet this holds true in our current assumptions of the network as well: that it will become a collection of pruned nodes with a few storage nodes. A hybrid option makes this better, because it spreads the risk, rather than concentrating it in full nodes. On May 12, 2015 3:38 PM, Jeff Garzik

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread Tier Nolan
On Tue, May 12, 2015 at 6:16 PM, Peter Todd p...@petertodd.org wrote: Lots of people are tossing around ideas for partial archival nodes that would store a subset of blocks, such that collectively the whole blockchain would be available even if no one node had the entire chain. A compact

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread Gregory Maxwell
It's a little frustrating to see this just repeated without even paying attention to the desirable characteristics from the prior discussions. Summarizing from memory: (0) Block coverage should have locality; historical blocks are (almost) always needed in contiguous ranges. Having random

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread Jeff Garzik
One general problem is that security is weakened when an attacker can DoS a small part of the chain by DoS'ing a small number of nodes - yet the impact is a network-wide DoS because nobody can complete a sync. On Tue, May 12, 2015 at 12:24 PM, gabe appleton gapplet...@gmail.com wrote: 0, 1, 3,

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread gabe appleton
0, 1, 3, 4, 5, 6 can be solved by looking at chunks chronologically. Ie, give the signed (by sender) hash of the first and last block in your range. This is less data dense than the idea above, but it might work better. That said, this is likely a less secure way to do it. To improve upon that, a

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread gabe appleton
I suppose this begs two questions: 1) why not have a partial archive store the most recent X% of the blockchain by default? 2) why not include some sort of torrent in QT, to mitigate this risk? I don't think this is necessarily a good idea, but I'd like to hear the reasoning. On May 12, 2015

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread Gregory Maxwell
On Tue, May 12, 2015 at 8:10 PM, Jeff Garzik jgar...@bitpay.com wrote: True. Part of the issue rests on the block sync horizon/cliff. There is a value X which is the average number of blocks the 90th percentile of nodes need in order to sync. It is sufficient for the [semi-]pruned nodes to

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread Tier Nolan
On Tue, May 12, 2015 at 8:03 PM, Gregory Maxwell gmaxw...@gmail.com wrote: (0) Block coverage should have locality; historical blocks are (almost) always needed in contiguous ranges. Having random peers with totally random blocks would be horrific for performance; as you'd have to hunt

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread gabe appleton
This is exactly the sort of solution I was hoping for. It seems this is the minimal modification to make it work, and, if someone was willing to work with me, I would love to help implement this. My only concern would be if the - - max-size flag is not included than this delivers significantly

Re: [Bitcoin-development] Proposed additional options for pruned nodes

2015-05-12 Thread Daniel Kraft
Hi all! On 2015-05-12 21:03, Gregory Maxwell wrote: Summarizing from memory: In the context of this discussion, let me also restate an idea I've proposed in Bitcointalk for this. It is probably not perfect and could surely be adapted (I'm interested in that), but I think it meets most/all of