Re: [Bitcoin-development] Why are we bleeding nodes?

2014-05-20 Thread bitcoingrant
Recently China has updated its firewall blocking bitcoin sites and pools. Whether this is simple blacklist or moresophisticatedpacket targeting is uncertain, however this update did spefically target VPN handshakes. Sent:Monday, April 07, 2014 at 1:07 PM From:Drak d...@zikula.org To:Mike

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-05-20 Thread Mike Hearn
...@plan99.net *Cc:* Bitcoin Dev bitcoin-development@lists.sourceforge.net *Subject:* Re: [Bitcoin-development] Why are we bleeding nodes? For what it's worth, the number of nodes rose dramatically during the China bullrun (I recall 45k in China alone) and dropped as dramatically as the price

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-05-20 Thread Eugen Leitl
On Tue, May 20, 2014 at 10:15:44AM +0200, bitcoingr...@gmx.com wrote: Recently China has updated its firewall blocking bitcoin sites and pools. Whether this is simple blacklist or more sophisticated packet targeting is uncertain, however this update did spefically target VPN

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-05-20 Thread Gmail
Unlikely. I doubt any significant portion of miners in china will continue to mine on a china-specific chain, since it will certainly be outmined by non-Chinese miners, and will be orphaned eventually. More likely is that mining interests in china will make special arrangements to circumvent

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-05-20 Thread Andy Alness
Has there ever been serious discussion on extending the protocol to support UDP transport? That would allow for NAT traversal and for many more people to run effective nodes. I'm also curious if it could be made improve block propagation time. On Tue, May 20, 2014 at 7:52 AM, Gmail

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-05-20 Thread Jeff Garzik
Yes, i spec'd out the UDP traversal of the P2P protocol. It seems reasonable especially for inv messages. On Tue, May 20, 2014 at 2:46 PM, Andy Alness a...@coinbase.com wrote: Has there ever been serious discussion on extending the protocol to support UDP transport? That would allow for NAT

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-05-20 Thread Isidor Zeuner
In my opinion, the number of full nodes doesn't matter (as long as it's enough to satisfy demand by other nodes). Correct. Still, a high number of nodes has a few other benefits: 1) The more nodes there are, the cheaper it should be to run each one, given that the bandwidth and CPU

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-05-20 Thread Andy Alness
Awesome! I'm assuming this is it: https://bitcointalk.org/index.php?topic=156769.0 It would be interesting (at least to me) to take this a step further and offer UDP as a full TCP replacement capable of STUN-assisted NAT traversal and possibly swarmed blockchain syncs. It would require open TCP

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-05-20 Thread Jeff Garzik
Indeed -- you must reinvent TCP over UDP, ultimately, to handle blocks and large TXs. On Tue, May 20, 2014 at 4:09 PM, Andy Alness a...@coinbase.com wrote: Awesome! I'm assuming this is it: https://bitcointalk.org/index.php?topic=156769.0 It would be interesting (at least to me) to take this

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 12:38 PM, Wendell w...@hivewallet.com wrote: On that note, I think we have every possibility to make desktop and mobile wallets mind-numbingly simple -- and perhaps even do one better. Is this now a community priority? If so, I would really appreciate some additional

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-08 Thread Jean-Paul Kogelman
Isn't that just conceding that p2p protocol A is better than p2p protocol B? Can't Bitcoin Core's block fetching be improved to get similar performance as a torrent + import? Currently it's hard to go wide on data fetching because headers first is still pretty 'beefy'. The headers can be

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-08 Thread Mike Hearn
Multi-sig requires infrastructure. It isn't a magic wand that we can wave to make everyone secure. The protocols and techniques necessary don't exist yet, and apparently no one has much of an incentive to create them. It is starting to happen. If you're OK with using a specific web wallet

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-08 Thread Tamas Blummer
Specialization of nodes is ongoing most prominent with SPV wallets and mining. There is a need I see on my own business for software that is able to serve multiple wallets, and is multi tiered, so the world facing P2P node can be in a DMZ. I target them with a hybrid model that is SPV plus

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-08 Thread Jesus Cea
On 07/04/14 15:50, Gregory Maxwell wrote: Bitcoin.org recommends people away from running Bitcoin-QT now, so I'm not sure that we should generally find that trend surprising. What options are out there for people caring about 20GB blockchain? Depending of third party server is not an option.

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-08 Thread Andrew LeCody
My node (based in Dallas, TX) has about 240 connections and is using a little under 4 Mbps in bandwidth right now. According the hosting provider I'm at 11.85 Mbps for this week, using 95th percentile billing. The report from my provider includes my other servers though. On Mon, Apr 7, 2014 at

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Jameson Lopp
I'm glad to see that I'm not the only one concerned about the consistent dropping of nodes. Though I think that the fundamental question should be: how many nodes do we really need? Obviously more is better, but it's difficult to say how concerned we should be without more information. I posted

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Pieter Wuille
On Mon, Apr 7, 2014 at 2:19 PM, Jameson Lopp jameson.l...@gmail.com wrote: I'm glad to see that I'm not the only one concerned about the consistent dropping of nodes. Though I think that the fundamental question should be: how many nodes do we really need? Obviously more is better, but it's

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Mike Hearn
In my opinion, the number of full nodes doesn't matter (as long as it's enough to satisfy demand by other nodes). Correct. Still, a high number of nodes has a few other benefits: 1) The more nodes there are, the cheaper it should be to run each one, given that the bandwidth and CPU for

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Jameson Lopp
On 04/07/2014 08:26 AM, Pieter Wuille wrote: In my opinion, the number of full nodes doesn't matter (as long as it's enough to satisfy demand by other nodes). I agree, but if we don't quantify demand then we are practically blind. What is the plan? To wait until SPV clients start lagging /

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Andreas Schildbach
They're _not_ phasing out into SPV wallets from what I can say. At around the 24th of February, there has been a sharp change of the current installs graph of Bitcoin Wallet. That number used to grow at about 20.000 per month. After that date until now, it just barely moves horizontal. My guess

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 4:34 AM, Mike Hearn m...@plan99.net wrote: At the start of February we had 10,000 bitcoin nodes. Now we have 8,500 and still falling: FWIW, A few months before that we had even less than 8500 by the bitnodes count. Bitcoin.org recommends people away from running

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 6:50 AM, Gregory Maxwell gmaxw...@gmail.com wrote: FWIW, A few months before that we had even less than 8500 by the bitnodes count. Gah, accidentally send I wanted to continue here that it was less than 8500 and had been falling pretty consistently for months,

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Jameson Lopp
The Bitnodes project updated their counting algorithm a month or so ago. It used to be slower and less accurate - prior to their update, it was reporting in excess of 100,000 nodes. - Jameson On 04/07/2014 09:53 AM, Gregory Maxwell wrote: On Mon, Apr 7, 2014 at 6:50 AM, Gregory Maxwell

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 6:58 AM, Jameson Lopp jameson.l...@gmail.com wrote: The Bitnodes project updated their counting algorithm a month or so ago. It used to be slower and less accurate - prior to their update, it was reporting in excess of 100,000 nodes. Nah. It reported multiple metrics.

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Mike Hearn
My guess is that a large number of users have lost interest after they lost their money in MtGox. The 24th of February coincides with the final shutdown Sigh. It would not be surprising if MtGox has indeed dealt the community a critical blow in this regard. TX traffic is down since then too:

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Mike Hearn
Indeed, fully agreed. The only way to really make progress here is to make the UX of being your own bank not only as good as trusting a third party, but better. I've been encouraged by the rise of risk analysis services, but we need to integrate them into wallets more widely for them to have much

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Eric Martindale
We need to make it so mind-numbingly simple to run Bitcoin correctly that the average user doesn't find reasons to do so in the course of normal use. Right now, Coinbase and Bitstamp are winning in the user experience battle, which technically endanger the user, and by proxy the Bitcoin network.

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Jameson Lopp
I would point to bandwidth as the most important issue to the casual user who runs a node at home. Few casual users have the know-how to set up QoS rules and thus become quite annoyed when their Internet connection is discernibly slowed. - Jameson On 04/07/2014 11:53 AM, Gregory Maxwell wrote:

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Mark Friedenbach
Right now running a full-node on my home DSL connection (1Mbps) makes other internet activity periodically unresponsive. I think we've already hit a point where resource requirements are pushing out casual users, although of course we can't be certain that accounts for all lost nodes. On

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 9:27 AM, Mark Friedenbach m...@monetize.io wrote: Right now running a full-node on my home DSL connection (1Mbps) makes other internet activity periodically unresponsive. I think we've already hit a point where resource requirements are pushing out casual users, although

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Drak
For what it's worth, the number of nodes rose dramatically during the China bullrun (I recall 45k in China alone) and dropped as dramatically as the price after the first PBOC announcement designed to cool down bitcoin trading in China. On 7 April 2014 12:34, Mike Hearn m...@plan99.net wrote:

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 10:01 AM, Mark Friedenbach m...@monetize.io wrote: On 04/07/2014 09:57 AM, Gregory Maxwell wrote: That is an implementation issue— mostly one that arises as an indirect consequence of not having headers first and the parallel fetch, not a requirements issue. Oh,

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Mark Friedenbach
On 04/07/2014 09:57 AM, Gregory Maxwell wrote: That is an implementation issue— mostly one that arises as an indirect consequence of not having headers first and the parallel fetch, not a requirements issue. Oh, absolutely. But the question why are people not running full nodes? has to do with

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Brent Shambaugh
How difficult would it be to set up a node? Using lots of electricity at home (if required) could be an issue, but I do have a Webfaction account. On Mon, Apr 7, 2014 at 12:16 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Mon, Apr 7, 2014 at 10:01 AM, Mark Friedenbach m...@monetize.io

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Mike Hearn
It uses ~no electricity, it's not like mining. The primary resources it needs are disk space and bandwidth, after an intensive initial day or two of building the database. Actually, I wonder if we should start shipping (auditable) pre-baked databases calculated up to the last checkpoint so

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 10:40 AM, Mike Hearn m...@plan99.net wrote: Actually, I wonder The actual validation isn't really the problem today. The slowness of the IBD is almost exclusively the lack of parallel fetching and the existence of slow peers. And making the signature gate adaptive (and

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tamas Blummer
I rather prefer to start with SPV and upgrade to full node, if desired. Tamas Blummer http://bitsofproof.com On 07.04.2014, at 19:40, Mike Hearn m...@plan99.net wrote: Actually, I wonder if we should start shipping (auditable) pre-baked databases calculated up to the last checkpoint so

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Justus Ranvier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/07/2014 05:16 PM, Gregory Maxwell wrote: When I read resource requirements of a full node are moving beyond I didn't extract from that that there are implementation issues that need to be improved to make it work better for low resource

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Brent Shambaugh
Okay awesome. It seems like I set up a Litecoin node without knowing it (because it was like this: https://bitcointalk.org/index.php?topic=128122.0) I was able to bootstrap it (https://litecoin.info/). On Mon, Apr 7, 2014 at 12:40 PM, Mike Hearn m...@plan99.net wrote: It uses ~no electricity,

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Chris Williams
I’m afraid this is a highly simplistic view of the costs of running a full node. My node consumes fantastic amounts of data traffic, which is a real cost. In the 30 days ending Apri 6, my node: * Received 36.8 gb of data * Sent 456.5 gb data At my geographic service location (Singapore), this

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Justus Ranvier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/07/2014 05:40 PM, Mike Hearn wrote: The primary resources it needs are disk space and bandwidth, after an intensive initial day or two of building the database. Check out the kind of hardware causal users are running these days. The

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Mike Hearn
* Sent 456.5 gb data At my geographic service location (Singapore), this cost about $90 last month for bandwidth alone. One of the reasons I initiated the (now stalled) PayFile project was in anticipation of this problem: https://github.com/mikehearn/PayFile

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tamas Blummer
Headers first loading allows the node to run SPV from the very first minutes and it can converge to full node by time. This is BTW how newest versions of BOP can work. Pruning however disqualifies the node as a source for bootstrapping an other full node. BTW, did we already agree on the

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 11:35 AM, Tamas Blummer ta...@bitsofproof.com wrote: BTW, did we already agree on the service bits for an archive node? I'm still very concerned that a binary archive bit will cause extreme load hot-spotting and the kind of binary Use lots of resources YES or NO I think

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tamas Blummer
Once a single transaction in pruned in a block, the block is no longer eligible to be served to other nodes. Which transactions are pruned can be rather custom e.g. even depending on the wallet(s) of the node, therefore I guess it is more handy to return some bitmap of pruned/full blocks than

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com wrote: Once a single transaction in pruned in a block, the block is no longer eligible to be served to other nodes. Which transactions are pruned can be rather custom e.g. even depending on the wallet(s) of the node,

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com wrote: therefore I guess it is more handy to return some bitmap of pruned/full blocks than ranges. A bitmap also means high overhead and— if it's used to advertise non-contiguous blocks— poor locality, since blocks are fetched

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Mark Friedenbach
On 04/07/2014 12:00 PM, Tamas Blummer wrote: Once a single transaction in pruned in a block, the block is no longer eligible to be served to other nodes. Which transactions are pruned can be rather custom e.g. even depending on the wallet(s) of the node, therefore I guess it is more handy to

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tamas Blummer
Maybe it is not a question of the maturity of the implementation but that of the person making presumptions of it. I consider a fully pruned blockchain being equivalent to the UTXO. Block that hold no more unspent transaction are reduced to a header. There is however no harm if more retained.

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tamas Blummer
Once headers are loaded first there is no reason for sequential loading. Validation has to be sequantial, but that step can be deferred until the blocks before a point are loaded and continous. Tamas Blummer http://bitsofproof.com On 07.04.2014, at 21:03, Gregory Maxwell gmaxw...@gmail.com

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Mark Friedenbach
On 04/07/2014 12:20 PM, Tamas Blummer wrote: Validation has to be sequantial, but that step can be deferred until the blocks before a point are loaded and continous. And how do you find those blocks? I have a suggestion: have nodes advertise which range of full blocks they possess, then you

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tamas Blummer
You have the trunk defined by the headers. Once a range from genesis to block n is fully downloaded, you may validate upto block n. Furthermore after validation you can prune transactions spent until block n. You would approach the highest block with validation and stop pruning say 100 blocks

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Paul Lyon
...@gmail.com CC: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] Why are we bleeding nodes? Once headers are loaded first there is no reason for sequential loading. Validation has to be sequantial, but that step can be deferred until the blocks before a point are loaded

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Troy Benjegerdes
I understand the theoretical benefits of multi-sig. But if you want to make this mind-numbingly simple, do it on the *existing* single-sig. But why in the world do we not have a *business* that offers bitcoin wallet insurance? The bitcoin world (and this list) ran around blaming MtGox and users

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tamas Blummer
have any critical design flaws in there. :) From: ta...@bitsofproof.com Date: Mon, 7 Apr 2014 21:20:31 +0200 To: gmaxw...@gmail.com CC: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] Why are we bleeding nodes? Once headers are loaded first there is no reason

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Ricardo Filipe
Or have blocks distributed through pruned nodes as a DHT. 2014-04-07 20:13 GMT+01:00 Mark Friedenbach m...@monetize.io: On 04/07/2014 12:20 PM, Tamas Blummer wrote: Validation has to be sequantial, but that step can be deferred until the blocks before a point are loaded and continous. And

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tier Nolan
learning exercise for me, so I can only hope I don't have any critical design flaws in there. :) -- From: ta...@bitsofproof.com Date: Mon, 7 Apr 2014 21:20:31 +0200 To: gmaxw...@gmail.com CC: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] Why

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Gregory Maxwell
On Mon, Apr 7, 2014 at 2:48 PM, Tier Nolan tier.no...@gmail.com wrote: Blocks can be loaded in random order once you have their order given by the headers. Computing the UTXO however will force you to at least temporarily store the blocks unless you have plenty of RAM. You only need to store

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tier Nolan
...@gmail.com CC: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] Why are we bleeding nodes? Once headers are loaded first there is no reason for sequential loading. Validation has to be sequantial, but that step can be deferred until the blocks before a point are loaded

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread kjj
Multi-sig requires infrastructure. It isn't a magic wand that we can wave to make everyone secure. The protocols and techniques necessary don't exist yet, and apparently no one has much of an incentive to create them. I mean no offense, and I don't mean to pick on you. Your post stuck out

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Jeff Garzik
Being Mr. Torrent, I've held open the 80% serious suggestion to simply refuse to serve blocks older than X (3 months?). That forces download by other means (presumably torrent). I do not feel it is productive for any nodes on the network to waste time/bandwidth/etc. serving static, ancient data.