subject:"\[Bitcoin\-development\] Why are we bleeding nodes\?"

I'm glad to see that I'm not the only one concerned about the consistent 
dropping of nodes. Though I think that the fundamental question should be: how 
many nodes do we really need? Obviously more is better, but it's difficult to 
say how concerned we should be without more information. I posted my thoughts 
last month: http://coinchomp.com/2014/03/19/bitcoin-nodes-many-enough/

I have begun working on my node monitoring project and will post updates if it 
results in me gaining any new insights about the network.

- Jameson

On 04/07/2014 07:34 AM, Mike Hearn wrote:
 At the start of February we had 10,000 bitcoin nodes. Now we have 8,500 and
 still falling:
 
http://getaddr.bitnodes.io/dashboard/chart/?days=60
 
 I know all the reasons why people *might* stop running a node (uses too
 much disk space, bandwidth, lost interest etc). But does anyone have any
 idea how we might get more insight into what's really going on? It'd be
 convenient if the subVer contained the operating system, as then we could
 tell if the bleed was mostly from desktops/laptops (Windows/Mac), which
 would be expected, or from virtual servers (Linux), which would be more
 concerning.
 
 When you set up a Tor node, you can add your email address to the config
 file and the Tor project sends you emails from time to time about things
 you should know about. If we did the same, we could have a little exit
 survey: if your node disappears for long enough, we could email the
 operator and ask why they stopped.
 
 
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees_APR
 
 
 
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development
 

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Pieter Wuille

On Mon, Apr 7, 2014 at 2:19 PM, Jameson Lopp jameson.l...@gmail.com wrote:
 I'm glad to see that I'm not the only one concerned about the consistent 
 dropping of nodes. Though I think that the fundamental question should be: 
 how many nodes do we really need? Obviously more is better, but it's 
 difficult to say how concerned we should be without more information. I 
 posted my thoughts last month: 
 http://coinchomp.com/2014/03/19/bitcoin-nodes-many-enough/

In my opinion, the number of full nodes doesn't matter (as long as
it's enough to satisfy demand by other nodes).

What matters is how hard it is to run one. If someone is interesting
in verifying that nobody is cheating on the network, can they, and can
they without significant investment? Whether they actually will
depends also no how interesting the currency and its digital transfers
are.

 On 04/07/2014 07:34 AM, Mike Hearn wrote:
 At the start of February we had 10,000 bitcoin nodes. Now we have 8,500 and
 still falling:

http://getaddr.bitnodes.io/dashboard/chart/?days=60

My own network crawler (which feeds my DNS seeder) hasn't seen any
significant drop that I remember, but I don't have actual logs. It's
seeing around 6000 well reachable nodes currently, which is the
highest number I've ever seen (though it's been around 6000 for quite
a while now).

-- 
Pieter

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?


 In my opinion, the number of full nodes doesn't matter (as long as
 it's enough to satisfy demand by other nodes).


Correct. Still, a high number of nodes has a few other benefits:

1) The more nodes there are, the cheaper it should be to run each one,
given that the bandwidth and CPU for serving the chain will be spread over
more people.

2) It makes Bitcoin *seem* bigger, more robust and more decentralised,
because there are more people uniting to run it. So there's a psychological
benefit.

Also, we don't have a good way to measure capacity vs demand at the moment.
Whether we have enough capacity is rather a shot in the dark right now.


 What matters is how hard it is to run one.


Which is why I'm interested to learn the reason behind the drop. Is it
insufficient interest, or is running a node too painful?

For this purpose I'd like to exclude people running Bitcoin Core on laptops
or non-dedicated desktops. I don't think full nodes will ever make sense
for consumer wallets again, and I see the bleeding off of those people as
natural and expected (as Satoshi did). But if someone feels it's too hard
to run on a cheap server then that'd concern me.


 My own network crawler (which feeds my DNS seeder) hasn't seen any
 significant drop


It would be good to explain the difference, but I suspect your definition
of well reachable excludes people running Core at home. From the diurnal
cycle we see in Addy's graphs it's clear some nodes are being shut down
when people go to bed. So if we have 6000 nodes on servers and 2000 at
home, then I'd expect Addy's graphs and yours to slowly come into alignment
as people give up using Core as a consumer wallet.
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On 04/07/2014 08:26 AM, Pieter Wuille wrote:
 In my opinion, the number of full nodes doesn't matter (as long as
 it's enough to satisfy demand by other nodes).

I agree, but if we don't quantify demand then we are practically blind. What 
is the plan? To wait until SPV clients start lagging / timing out because their 
requests cannot be handled by the nodes?

For all I know, the network would run just fine on 100 nodes. But not knowing 
really irks me as an engineer.

- Jameson

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Andreas Schildbach

They're _not_ phasing out into SPV wallets from what I can say. At
around the 24th of February, there has been a sharp change of the
current installs graph of Bitcoin Wallet. That number used to grow at
about 20.000 per month. After that date until now, it just barely moves
horizontal.

My guess is that a large number of users have lost interest after they
lost their money in MtGox. The 24th of February coincides with the
final shutdown, according to

http://en.wikipedia.org/wiki/Mt._Gox#February_2014_shutdown_and_bankruptcy


On 04/07/2014 02:17 PM, Ricardo Filipe wrote:
 phasing out of bitcoinqt into spv wallets?
 
 2014-04-07 12:34 GMT+01:00 Mike Hearn m...@plan99.net:
 At the start of February we had 10,000 bitcoin nodes. Now we have 8,500 and
 still falling:

http://getaddr.bitnodes.io/dashboard/chart/?days=60

 I know all the reasons why people might stop running a node (uses too much
 disk space, bandwidth, lost interest etc). But does anyone have any idea how
 we might get more insight into what's really going on? It'd be convenient if
 the subVer contained the operating system, as then we could tell if the
 bleed was mostly from desktops/laptops (Windows/Mac), which would be
 expected, or from virtual servers (Linux), which would be more concerning.

 When you set up a Tor node, you can add your email address to the config
 file and the Tor project sends you emails from time to time about things you
 should know about. If we did the same, we could have a little exit survey:
 if your node disappears for long enough, we could email the operator and ask
 why they stopped.

 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees_APR
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development

 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees_APR
 


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On Mon, Apr 7, 2014 at 4:34 AM, Mike Hearn m...@plan99.net wrote:
 At the start of February we had 10,000 bitcoin nodes. Now we have 8,500 and
 still falling:

FWIW, A few months before that we had even less than 8500 by the bitnodes count.

Bitcoin.org recommends people away from running Bitcoin-QT now, so I'm
not sure that we should generally find that trend surprising.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On Mon, Apr 7, 2014 at 6:50 AM, Gregory Maxwell gmaxw...@gmail.com wrote:
 FWIW, A few months before that we had even less than 8500 by the bitnodes 
 count.

Gah, accidentally send I wanted to continue here that it was less
than 8500 and had been falling pretty consistently for months,
basically since the bitcoin.org change.  Unfortunately it looks like
the old bitnodes.io data isn't available anymore, so I'm going off my
memory here.

The Bitnodes counts have always been somewhat higher than my or sipa's
node counts too, fwiw.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

The Bitnodes project updated their counting algorithm a month or so ago. It 
used to be slower and less accurate - prior to their update, it was reporting 
in excess of 100,000 nodes.

- Jameson

On 04/07/2014 09:53 AM, Gregory Maxwell wrote:
 On Mon, Apr 7, 2014 at 6:50 AM, Gregory Maxwell gmaxw...@gmail.com wrote:
 FWIW, A few months before that we had even less than 8500 by the bitnodes 
 count.
 
 Gah, accidentally send I wanted to continue here that it was less
 than 8500 and had been falling pretty consistently for months,
 basically since the bitcoin.org change.  Unfortunately it looks like
 the old bitnodes.io data isn't available anymore, so I'm going off my
 memory here.
 
 The Bitnodes counts have always been somewhat higher than my or sipa's
 node counts too, fwiw.
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees_APR
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development
 

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On Mon, Apr 7, 2014 at 6:58 AM, Jameson Lopp jameson.l...@gmail.com wrote:
 The Bitnodes project updated their counting algorithm a month or so ago. It 
 used to be slower and less accurate - prior to their update, it was reporting 
 in excess of 100,000 nodes.

Nah.  It reported multiple metrics. The 100,000 number was an mostly
useless metric that just counted the number of distinct addr messages
floating around the network, which contains a lot of junk.  They also
reported an actual connectable node count previously and while they
may have tweaked things here and there as far as I can tell it has
been consistent with the numbers they are using in the headlines now.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

My guess is that a large number of users have lost interest after they
lost their money in MtGox. The 24th of February coincides with the
final shutdown

Sigh. It would not be surprising if MtGox has indeed dealt the community a
critical blow in this regard. TX traffic is down since then too:

https://blockchain.info/charts/n-transactions-excluding-popular?timespan=60daysshowDataPoints=falsedaysAverageString=1show_header=truescale=0address=

Judging from comments and the leaked user db, it seems a lot of well known
people lost money there (not me fortunately). I wish I could say people
have learned but from the size of the deposit base at Bitstamp they clearly
have not. A lot of Bitcoin users don't seem to be ready to be their own
bank, yet still want to own some on the assumption everyone else either is
or soon will be. So it's really only a matter of time until something goes
wrong with some large bitbank again, either Bitstamp or Coinbase.

Some days I wonder if Bitcoin will be killed off by people who just refuse
to use it properly before it ever gets a chance to shine. The general
public doesn't distinguish between Bitcoin users who deposit with a third
party and the real Bitcoin users who don't.
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

Indeed, fully agreed. The only way to really make progress here is to make
the UX of being your own bank not only as good as trusting a third party,
but better.

I've been encouraged by the rise of risk analysis services, but we need to
integrate them into wallets more widely for them to have much impact.
Otherwise people get to pick between a variety of wallets, none of which
have *all* the features they want. And TREZOR is cool, albeit, something
that's going to be for committed users only.

On Mon, Apr 7, 2014 at 4:15 PM, Eric Martindale e...@ericmartindale.comwrote:

We need to make it so mind-numbingly simple to run Bitcoin correctly
that the average user doesn't find reasons to do so in the course of normal
use. Right now, Coinbase and Bitstamp are winning in the user experience
battle, which technically endanger the user, and by proxy the Bitcoin
network.

Multi-sig as a default is a start. It won't succeed unless the user
experience is simply better than trusted third parties, but we need to
start the education process with the very basic fundamental: trusting a
third-party with full access to your Bitcoin is just replacing one
centralized banking system with another.

Eric Martindale
Developer Evangelist, BitPay
+1 (919) 374-2020
On Apr 7, 2014 7:05 AM, Mike Hearn m...@plan99.net wrote:

My guess is that a large number of users have lost interest after they
lost their money in MtGox. The 24th of February coincides with the
final shutdown

Sigh. It would not be surprising if MtGox has indeed dealt the community
a critical blow in this regard. TX traffic is down since then too:

https://blockchain.info/charts/n-transactions-excluding-popular?timespan=60daysshowDataPoints=falsedaysAverageString=1show_header=truescale=0address=

Judging from comments and the leaked user db, it seems a lot of well
known people lost money there (not me fortunately). I wish I could say
people have learned but from the size of the deposit base at Bitstamp they
clearly have not. A lot of Bitcoin users don't seem to be ready to be their
own bank, yet still want to own some on the assumption everyone else either
is or soon will be. So it's really only a matter of time until something
goes wrong with some large bitbank again, either Bitstamp or Coinbase.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Eric Martindale

We need to make it so mind-numbingly simple to run Bitcoin correctly that
the average user doesn't find reasons to do so in the course of normal
use. Right now, Coinbase and Bitstamp are winning in the user experience
battle, which technically endanger the user, and by proxy the Bitcoin
network.

Eric Martindale
Developer Evangelist, BitPay
+1 (919) 374-2020
On Apr 7, 2014 7:05 AM, Mike Hearn m...@plan99.net wrote:

My guess is that a large number of users have lost interest after they
lost their money in MtGox. The 24th of February coincides with the
final shutdown

Sigh. It would not be surprising if MtGox has indeed dealt the community a
critical blow in this regard. TX traffic is down since then too:

https://blockchain.info/charts/n-transactions-excluding-popular?timespan=60daysshowDataPoints=falsedaysAverageString=1show_header=truescale=0address=

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

I would point to bandwidth as the most important issue to the casual user who 
runs a node at home. Few casual users have the know-how to set up QoS rules and 
thus become quite annoyed when their Internet connection is discernibly slowed.

- Jameson

On 04/07/2014 11:53 AM, Gregory Maxwell wrote:
 On Mon, Apr 7, 2014 at 8:45 AM, Justus Ranvier justusranv...@gmail.com 
 wrote:
 1. The resource requirements of a full node are moving beyond the
 capabilities of casual users. This isn't inherently a problem - after
 all most people don't grow their own food, tailor their own clothes, or
 keep blacksmith tools handy in to forge their own horseshoes either.
 
 Right now running a full node consumes about $1 in disk space
 non-reoccurring and costs a couple cents in power per month.
 
 This isn't to say things are all ducky. But if you're going to say the
 resource requirements are beyond the capabilities of casual users I'm
 afraid I'm going to have to say: citation needed.
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees_APR
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development
 

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

Right now running a full-node on my home DSL connection (1Mbps) makes
other internet activity periodically unresponsive. I think we've already
hit a point where resource requirements are pushing out casual users,
although of course we can't be certain that accounts for all lost nodes.

On 04/07/2014 08:53 AM, Gregory Maxwell wrote:
 On Mon, Apr 7, 2014 at 8:45 AM, Justus Ranvier justusranv...@gmail.com 
 wrote:
 1. The resource requirements of a full node are moving beyond the
 capabilities of casual users. This isn't inherently a problem - after
 all most people don't grow their own food, tailor their own clothes, or
 keep blacksmith tools handy in to forge their own horseshoes either.
 
 Right now running a full node consumes about $1 in disk space
 non-reoccurring and costs a couple cents in power per month.
 
 This isn't to say things are all ducky. But if you're going to say the
 resource requirements are beyond the capabilities of casual users I'm
 afraid I'm going to have to say: citation needed.
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees_APR
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development
 .
 

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On Mon, Apr 7, 2014 at 9:27 AM, Mark Friedenbach m...@monetize.io wrote:
 Right now running a full-node on my home DSL connection (1Mbps) makes
 other internet activity periodically unresponsive. I think we've already
 hit a point where resource requirements are pushing out casual users,
 although of course we can't be certain that accounts for all lost nodes.

That is an implementation issue— mostly one that arises as an indirect
consequence of not having headers first and the parallel fetch, not a
requirements issue.

Under the current bitcoin validity rules it should be completely
reasonable to run a full contributing node with no more than 30 kb/s
inbound (reviving two copies of everything, blocks + tansactions ) and
60 kbit/sec outbound (sending out four copies of everything). (So long
as you're sending out = what you're taking in you're contributing to
the network's capacity). Throw in a factor of two for bursting, though
not every node needs to be contributing super low latency capacity.

This is absolutely not the case with the current implementation, but
it's not a requirements thing.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Drak

For what it's worth, the number of nodes rose dramatically during the China
bullrun (I recall 45k in China alone) and dropped as dramatically as the
price after the first PBOC announcement designed to cool down bitcoin
trading in China.


On 7 April 2014 12:34, Mike Hearn m...@plan99.net wrote:

 At the start of February we had 10,000 bitcoin nodes. Now we have 8,500
 and still falling:

http://getaddr.bitnodes.io/dashboard/chart/?days=60

 I know all the reasons why people *might* stop running a node (uses too
 much disk space, bandwidth, lost interest etc). But does anyone have any
 idea how we might get more insight into what's really going on? It'd be
 convenient if the subVer contained the operating system, as then we could
 tell if the bleed was mostly from desktops/laptops (Windows/Mac), which
 would be expected, or from virtual servers (Linux), which would be more
 concerning.

 When you set up a Tor node, you can add your email address to the config
 file and the Tor project sends you emails from time to time about things
 you should know about. If we did the same, we could have a little exit
 survey: if your node disappears for long enough, we could email the
 operator and ask why they stopped.


 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees_APR
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On Mon, Apr 7, 2014 at 10:01 AM, Mark Friedenbach m...@monetize.io wrote:
 On 04/07/2014 09:57 AM, Gregory Maxwell wrote:
 That is an implementation issue— mostly one that arises as an indirect
 consequence of not having headers first and the parallel fetch, not a
 requirements issue.

 Oh, absolutely. But the question why are people not running full
 nodes? has to do with the current implementation, not abstract
 capabilities of a future version of the bitcoind code base.

The distinction is very important because it's a matter of things we
can and should fix vs things that cannot be fixed except by changing
goals/incentives!  Opposite approaches to handling them.

When I read resource requirements of a full node are moving beyond I
didn't extract from that that there are implementation issues that
need to be improved to make it work better for low resource users due
to the word requirements.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On 04/07/2014 09:57 AM, Gregory Maxwell wrote:
 That is an implementation issue— mostly one that arises as an indirect
 consequence of not having headers first and the parallel fetch, not a
 requirements issue.

Oh, absolutely. But the question why are people not running full
nodes? has to do with the current implementation, not abstract
capabilities of a future version of the bitcoind code base.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Brent Shambaugh

How difficult would it be to set up a node? Using lots of electricity at
home (if required) could be an issue, but I do have a Webfaction account.


On Mon, Apr 7, 2014 at 12:16 PM, Gregory Maxwell gmaxw...@gmail.com wrote:

 On Mon, Apr 7, 2014 at 10:01 AM, Mark Friedenbach m...@monetize.io
 wrote:
  On 04/07/2014 09:57 AM, Gregory Maxwell wrote:
  That is an implementation issue-- mostly one that arises as an indirect
  consequence of not having headers first and the parallel fetch, not a
  requirements issue.
 
  Oh, absolutely. But the question why are people not running full
  nodes? has to do with the current implementation, not abstract
  capabilities of a future version of the bitcoind code base.

 The distinction is very important because it's a matter of things we
 can and should fix vs things that cannot be fixed except by changing
 goals/incentives!  Opposite approaches to handling them.

 When I read resource requirements of a full node are moving beyond I
 didn't extract from that that there are implementation issues that
 need to be improved to make it work better for low resource users due
 to the word requirements.


 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

It uses ~no electricity, it's not like mining.

The primary resources it needs are disk space and bandwidth, after an
intensive initial day or two of building the database.

Actually, I wonder if we should start shipping (auditable) pre-baked
databases calculated up to the last checkpoint so people can download them
and boot up their node right away. Recalculating the entire thing from
scratch every time isn't sustainable in the long run anyway.


On Mon, Apr 7, 2014 at 7:35 PM, Brent Shambaugh
brent.shamba...@gmail.comwrote:

 How difficult would it be to set up a node? Using lots of electricity at
 home (if required) could be an issue, but I do have a Webfaction account.


 On Mon, Apr 7, 2014 at 12:16 PM, Gregory Maxwell gmaxw...@gmail.comwrote:

 On Mon, Apr 7, 2014 at 10:01 AM, Mark Friedenbach m...@monetize.io
 wrote:
  On 04/07/2014 09:57 AM, Gregory Maxwell wrote:
  That is an implementation issue— mostly one that arises as an indirect
  consequence of not having headers first and the parallel fetch, not a
  requirements issue.
 
  Oh, absolutely. But the question why are people not running full
  nodes? has to do with the current implementation, not abstract
  capabilities of a future version of the bitcoind code base.

 The distinction is very important because it's a matter of things we
 can and should fix vs things that cannot be fixed except by changing
 goals/incentives!  Opposite approaches to handling them.

 When I read resource requirements of a full node are moving beyond I
 didn't extract from that that there are implementation issues that
 need to be improved to make it work better for low resource users due
 to the word requirements.


 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development




 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On Mon, Apr 7, 2014 at 10:40 AM, Mike Hearn m...@plan99.net wrote:
 Actually, I wonder

The actual validation isn't really the problem today. The slowness of
the IBD is almost exclusively the lack of parallel fetching and the
existence of slow peers.  And making the signature gate adaptive (and
deploying the 6x faster ECDSA code) would improve that future.

Go grab sipa's headers first branch, it has no problem saturating a
20mbit/sec pipe syncing up.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

I rather prefer to start with SPV and upgrade to full node, if desired.

Tamas Blummer
http://bitsofproof.com

On 07.04.2014, at 19:40, Mike Hearn m...@plan99.net wrote:

 
 Actually, I wonder if we should start shipping (auditable) pre-baked 
 databases calculated up to the last checkpoint so people can download them 
 and boot up their node right away. Recalculating the entire thing from 
 scratch every time isn't sustainable in the long run anyway.
 



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Justus Ranvier

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/07/2014 05:16 PM, Gregory Maxwell wrote:
 When I read resource requirements of a full node are moving
 beyond I didn't extract from that that there are implementation
 issues that need to be improved to make it work better for low
 resource users due to the word requirements.

In order to prevent future confusion: whenever I talk about
requirements (or generally use the present tense) , I'm talking about
reality as it currently exists.

If I ever decide to talk about hypothetical future requirements in
some imaginary world of Platonic forms, as opposed to the requirements
imposed by the software that's actually available for casual users to
download today, I'll mention that specifically.

- -- 
Support online privacy by using email encryption whenever possible.
Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTQuRkAAoJECoisBQbQ4v07zUH/2wNQ7+0211+wm5oQP/ABHg7
kQVeQcRz8/BhOp3hzv6HiQ9Oaekfz7QhClbJYPUPE3aR2gxGoCgT6B1G2N75q0pG
piGVgCeogaBA3Ny91sOPRMv92cGWpbTeyO+rhIIIlYWPiZobTaYttYYm1zF6oc6K
CdYzCW9X12/NIXxEkbPnAFJ01Uty0HKccTP+9jex7+gobzl2yCo4MyywwtWF9XEk
K9aZ3+3i+12+F4nDMAAimD02SV6dI8GMpahMf4kNXn0CcMefC9A28FdhhApPh7Nx
1phQnMPZMPdYt0aHgjgzt4N+SR/1EOUZaoqk9ccyXh+khBR84tiUEo6NuOIfTGM=
=Mr12
-END PGP SIGNATURE-


0x1B438BF4.asc
Description: application/pgp-keys
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Brent Shambaugh

Okay awesome. It seems like I set up a Litecoin node without knowing it
(because it was like this:
https://bitcointalk.org/index.php?topic=128122.0) I was able to bootstrap
it (https://litecoin.info/).


On Mon, Apr 7, 2014 at 12:40 PM, Mike Hearn m...@plan99.net wrote:

 It uses ~no electricity, it's not like mining.

 The primary resources it needs are disk space and bandwidth, after an
 intensive initial day or two of building the database.

 Actually, I wonder if we should start shipping (auditable) pre-baked
 databases calculated up to the last checkpoint so people can download them
 and boot up their node right away. Recalculating the entire thing from
 scratch every time isn't sustainable in the long run anyway.


 On Mon, Apr 7, 2014 at 7:35 PM, Brent Shambaugh brent.shamba...@gmail.com
  wrote:

 How difficult would it be to set up a node? Using lots of electricity at
 home (if required) could be an issue, but I do have a Webfaction account.


 On Mon, Apr 7, 2014 at 12:16 PM, Gregory Maxwell gmaxw...@gmail.comwrote:

 On Mon, Apr 7, 2014 at 10:01 AM, Mark Friedenbach m...@monetize.io
 wrote:
  On 04/07/2014 09:57 AM, Gregory Maxwell wrote:
  That is an implementation issue-- mostly one that arises as an indirect
  consequence of not having headers first and the parallel fetch, not a
  requirements issue.
 
  Oh, absolutely. But the question why are people not running full
  nodes? has to do with the current implementation, not abstract
  capabilities of a future version of the bitcoind code base.

 The distinction is very important because it's a matter of things we
 can and should fix vs things that cannot be fixed except by changing
 goals/incentives!  Opposite approaches to handling them.

 When I read resource requirements of a full node are moving beyond I
 didn't extract from that that there are implementation issues that
 need to be improved to make it work better for low resource users due
 to the word requirements.


 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development




 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development



--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Chris Williams

I’m afraid this is a highly simplistic view of the costs of running a full node.

My node consumes fantastic amounts of data traffic, which is a real cost.

In the 30 days ending Apri 6, my node:

* Received 36.8 gb of data
* Sent 456.5 gb data

At my geographic service location (Singapore), this cost about $90 last month 
for bandwidth alone. It would be slightly cheaper if I was hosted in the US of 
course.

But anyone can understand that moving a half-terrabyte of data around in a 
month will not be cheap.


On Apr 7, 2014, at 8:53 AM, Gregory Maxwell gmaxw...@gmail.com wrote:

 On Mon, Apr 7, 2014 at 8:45 AM, Justus Ranvier justusranv...@gmail.com 
 wrote:
 1. The resource requirements of a full node are moving beyond the
 capabilities of casual users. This isn't inherently a problem - after
 all most people don't grow their own food, tailor their own clothes, or
 keep blacksmith tools handy in to forge their own horseshoes either.
 
 Right now running a full node consumes about $1 in disk space
 non-reoccurring and costs a couple cents in power per month.
 
 This isn't to say things are all ducky. But if you're going to say the
 resource requirements are beyond the capabilities of casual users I'm
 afraid I'm going to have to say: citation needed.
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees_APR
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Justus Ranvier

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/07/2014 05:40 PM, Mike Hearn wrote:
 The primary resources it needs are disk space and bandwidth, after
 an intensive initial day or two of building the database.

Check out the kind of hardware causal users are running these days.

The bottleneck is not bulk disk space, but rather IOPS.

Most users don't have spare machines to dedicate to the task of
running a full node, nor is it acceptable for them to not be able to
use their device for other tasks while the node is bootstrapping.

- -- 
Support online privacy by using email encryption whenever possible.
Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTQuVNAAoJECoisBQbQ4v004EIAMPpQLrVlzCoGjcgALyHV4xK
6JDnlCHXTR72mTlKwcbD6Dpyr/Dl6tcXtdbQi0m3gsbOcAZI/eHtrswgunaq7c1y
mTOM2klPE4M8+B/4Ecp+2iK2UM/swlL3z8ryx/HPhgOZ+Rr7AENe3WUYOKiVNE4O
YQP1x9c0l09ma3ZU0sLmz2VTyqVzI7yV3mu/+HKcwYnqNrK9i/c8MRhOLdZ0gcUL
b2PtjjCdnSNLelZvwSLcqqR5+1oejAVwgt1Aq4RGyZzD9DVdCXUR9c9HWLAs0MEU
WPU5gU03YmrE5mkgGHRO3YnDbky0gdAGEw2Phzqd2upud4CLqdhEqA4v6/tQhpk=
=BVpU
-END PGP SIGNATURE-


0x1B438BF4.asc
Description: application/pgp-keys
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?


 * Sent 456.5 gb data

 At my geographic service location (Singapore), this cost about $90 last
 month for bandwidth alone.


One of the reasons I initiated the (now stalled) PayFile project was in
anticipation of this problem:

https://github.com/mikehearn/PayFile
http://www.youtube.com/watch?v=r0BXnWlnIi4feature=youtu.be

At some point if you want to actually download and validate the full block
chain from scratch, you will have to start paying for it I'm sure.

In the meantime:

   1. Getting headers-first implemented and rolled out everywhere would
   reduce the amount of redundant downloading and hopefully reduce transmit
   traffic network-wide.
   2. Implementing chain pruning would allow people to control upload
   bandwidth consumption by reducing the amount of disk storage they allow.
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

Headers first loading allows the node to run SPV from the very first minutes 
and it can converge to full node by time.
This is BTW how newest versions of BOP can work.

Pruning however disqualifies the node as a source for bootstrapping an other 
full node. 
BTW, did we already agree on the service bits for an archive node?


Tamas Blummer
http://bitsofproof.com

On 07.04.2014, at 20:23, Mike Hearn m...@plan99.net wrote:

 * Sent 456.5 gb data
 
 At my geographic service location (Singapore), this cost about $90 last month 
 for bandwidth alone.
 
 One of the reasons I initiated the (now stalled) PayFile project was in 
 anticipation of this problem:
 
 https://github.com/mikehearn/PayFile
 http://www.youtube.com/watch?v=r0BXnWlnIi4feature=youtu.be
 
 At some point if you want to actually download and validate the full block 
 chain from scratch, you will have to start paying for it I'm sure.
 
 In the meantime:
 Getting headers-first implemented and rolled out everywhere would reduce the 
 amount of redundant downloading and hopefully reduce transmit traffic 
 network-wide.
 Implementing chain pruning would allow people to control upload bandwidth 
 consumption by reducing the amount of disk storage they allow.
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On Mon, Apr 7, 2014 at 11:35 AM, Tamas Blummer ta...@bitsofproof.com wrote:
 BTW, did we already agree on the service bits for an archive node?

I'm still very concerned that a binary archive bit will cause extreme
load hot-spotting and the kind of binary Use lots of resources YES or
NO I think we're currently suffering some from, but at that point
enshrined in the protocol.

It would be much better to extend the addr messages so that nodes can
indicate a range or two of blocks that they're serving, so that all
nodes can contribute fractionally according to their means. E.g. if
you want to offer up 8 GB of distributed storage and contribute to the
availability of the blockchain,  without having to swollow the whole
20, 30, 40 ... gigabyte pill.

Already we need that kind of distributed storage for the most recent
blocks to prevent extreme bandwidth load on archives, so extending it
to arbitrary ranges is only more complicated because there is
currently no room to signal it.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

Once a single transaction in pruned in a block, the block is no longer eligible 
to be served to other nodes. 
Which transactions are pruned can be rather custom e.g. even depending on the 
wallet(s) of the node,
therefore I guess it is more handy to return some bitmap of pruned/full blocks 
than ranges.

Tamas Blummer
http://bitsofproof.com

On 07.04.2014, at 20:49, Gregory Maxwell gmaxw...@gmail.com wrote:

 On Mon, Apr 7, 2014 at 11:35 AM, Tamas Blummer ta...@bitsofproof.com wrote:
 BTW, did we already agree on the service bits for an archive node?
 
 I'm still very concerned that a binary archive bit will cause extreme
 load hot-spotting and the kind of binary Use lots of resources YES or
 NO I think we're currently suffering some from, but at that point
 enshrined in the protocol.
 
 It would be much better to extend the addr messages so that nodes can
 indicate a range or two of blocks that they're serving, so that all
 nodes can contribute fractionally according to their means. E.g. if
 you want to offer up 8 GB of distributed storage and contribute to the
 availability of the blockchain,  without having to swollow the whole
 20, 30, 40 ... gigabyte pill.
 
 Already we need that kind of distributed storage for the most recent
 blocks to prevent extreme bandwidth load on archives, so extending it
 to arbitrary ranges is only more complicated because there is
 currently no room to signal it.
 



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com wrote:
 Once a single transaction in pruned in a block, the block is no longer
 eligible to be served to other nodes.
 Which transactions are pruned can be rather custom e.g. even depending on
 the wallet(s) of the node,
 therefore I guess it is more handy to return some bitmap of pruned/full
 blocks than ranges.

This isn't at all how pruning works in Bitcoin-QT  (nor is it how I
expect pruning to work for any mature implementation). Pruning can
work happily on a whole block at a time basis regardless if all the
transactions in it are spent or not.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com wrote:
 therefore I guess it is more handy to return some bitmap of pruned/full
 blocks than ranges.

A bitmap also means high overhead and— if it's used to advertise
non-contiguous blocks— poor locality, since blocks are fetched
sequentially.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

On 04/07/2014 12:00 PM, Tamas Blummer wrote:
 Once a single transaction in pruned in a block, the block is no longer
 eligible to be served to other nodes. 
 Which transactions are pruned can be rather custom e.g. even depending
 on the wallet(s) of the node,
 therefore I guess it is more handy to return some bitmap of pruned/full
 blocks than ranges.

The point is that the node has decided not to prune transactions from
that block, so that it is capable of returning full blocks within that
range.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

Maybe it is not a question of the maturity of the implementation but that of 
the person making presumptions of it.

I consider a fully pruned blockchain being equivalent to the UTXO. Block that 
hold no
more unspent transaction are reduced to a header. There is however no harm if 
more retained.

Tamas Blummer
http://bitsofproof.com

On 07.04.2014, at 21:02, Gregory Maxwell gmaxw...@gmail.com wrote:

 On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com wrote:
 Once a single transaction in pruned in a block, the block is no longer
 eligible to be served to other nodes.
 Which transactions are pruned can be rather custom e.g. even depending on
 the wallet(s) of the node,
 therefore I guess it is more handy to return some bitmap of pruned/full
 blocks than ranges.
 
 This isn't at all how pruning works in Bitcoin-QT  (nor is it how I
 expect pruning to work for any mature implementation). Pruning can
 work happily on a whole block at a time basis regardless if all the
 transactions in it are spent or not.
 



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?


Once headers are loaded first there is no reason for sequential loading. 

Validation has to be sequantial, but that step can be deferred until the blocks 
before a point are loaded and continous.

Tamas Blummer
http://bitsofproof.com

On 07.04.2014, at 21:03, Gregory Maxwell gmaxw...@gmail.com wrote:

 On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com wrote:
 therefore I guess it is more handy to return some bitmap of pruned/full
 blocks than ranges.
 
 A bitmap also means high overhead and— if it's used to advertise
 non-contiguous blocks— poor locality, since blocks are fetched
 sequentially.
 



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?



On 04/07/2014 12:20 PM, Tamas Blummer wrote:
 Validation has to be sequantial, but that step can be deferred until the
 blocks before a point are loaded and continous.

And how do you find those blocks?

I have a suggestion: have nodes advertise which range of full blocks
they possess, then you can perform synchronization from the adversed ranges!

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

You have the trunk defined by the headers. Once a range from genesis to block n 
is fully downloaded,
you may validate upto block n. Furthermore after validation you can prune 
transactions spent until block n.

You would approach the highest block with validation and stop pruning say 100 
blocks before it, to leave room for reorgs.

Tamas Blummer
http://bitsofproof.com

On 07.04.2014, at 21:13, Mark Friedenbach m...@monetize.io wrote:

 
 
 On 04/07/2014 12:20 PM, Tamas Blummer wrote:
 Validation has to be sequantial, but that step can be deferred until the
 blocks before a point are loaded and continous.
 
 And how do you find those blocks?
 
 I have a suggestion: have nodes advertise which range of full blocks
 they possess, then you can perform synchronization from the adversed ranges!
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development
 



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Paul Lyon

I hope I'm not thread-jacking here, apologies if so, but that's the approach 
I've taken with the node I'm working on.
Headers can be downloaded and stored in any order, it'll make sense of what the 
winning chain is. Blocks don't need to be downloaded in any particular order 
and they don't need to be saved to disk, the UTXO is fully self-contained. That 
way the concern of storing blocks for seeding (or not) is wholly separated from 
syncing the UTXO. This allows me to do the initial blockchain sync in ~6 hours 
when I use my SSD. I only need enough disk space to store the UTXO, and then 
whatever amount of block data the user would want to store for the health of 
the network.
This project is a bitcoin learning exercise for me, so I can only hope I don't 
have any critical design flaws in there. :)

From: ta...@bitsofproof.com
Date: Mon, 7 Apr 2014 21:20:31 +0200
To: gmaxw...@gmail.com
CC: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Why are we bleeding nodes?


Once headers are loaded first there is no reason for sequential loading. 
Validation has to be sequantial, but that step can be deferred until the blocks 
before a point are loaded and continous.
Tamas Blummerhttp://bitsofproof.com


On 07.04.2014, at 21:03, Gregory Maxwell gmaxw...@gmail.com wrote:On Mon, Apr 
7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com wrote:
therefore I guess it is more handy to return some bitmap of pruned/full
blocks than ranges.

A bitmap also means high overhead and— if it's used to advertise
non-contiguous blocks— poor locality, since blocks are fetched
sequentially.



--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
  --
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Troy Benjegerdes

I understand the theoretical benefits of multi-sig. But if you want
to make this mind-numbingly simple, do it on the *existing* single-sig.

But why in the world do we not have a *business* that offers bitcoin
wallet insurance? The bitcoin world (and this list) ran around blaming
MtGox and users for being 'stupid' to trust mtgox.

So start a multi-level marketing business that offers *insurance* so
if your bitcoin wallet gets hacked/stolen/whatever, your 'upstream'
or whomever sold you the wallet comes to your house with a new
computer or installs the new wallet software, or whatever, or just
makes it good.

Now, if the **insurance underwriter** decides that multisig will
reduce fraud, and **tests it**, then I'd say we do multi-sig. But right
now we are just a bunch of technology wizards trying to force our own
opinions about what's right and 'simple' for end users without ever
asking the damn end-users.

And then we call the end-users idiots because some scammer calls them
and says I'm calling from Microsoft and your computer is broke, please
download this software to fix it.

Multi-sig is more magical moon-math that scammers will exploit to con
your grandma out of bitcoin, and then your friends will call her a stupid
luddite for falling for it.

Fix the cultural victim-blaming bullshit and you'll fix the node bleeding
problem.

On Mon, Apr 07, 2014 at 10:15:15AM -0400, Eric Martindale wrote:
We need to make it so mind-numbingly simple to run Bitcoin correctly that
the average user doesn't find reasons to do so in the course of normal
use. Right now, Coinbase and Bitstamp are winning in the user experience
battle, which technically endanger the user, and by proxy the Bitcoin
network.

Eric Martindale
Developer Evangelist, BitPay
+1 (919) 374-2020
On Apr 7, 2014 7:05 AM, Mike Hearn m...@plan99.net wrote:

My guess is that a large number of users have lost interest after they
lost their money in MtGox. The 24th of February coincides with the
final shutdown

Sigh. It would not be surprising if MtGox has indeed dealt the community a
critical blow in this regard. TX traffic is down since then too:

https://blockchain.info/charts/n-transactions-excluding-popular?timespan=60daysshowDataPoints=falsedaysAverageString=1show_header=truescale=0address=

___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Troy Benjegerdes 'da hozer' ho...@hozed.org
7 elements earth::water::air::fire::mind::spirit::soulgrid.coop

Never pick a fight with someone who buys ink by the barrel,
nor try buy a hacker who makes money by the megahash

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration

Re: [Bitcoin-development] Why are we bleeding nodes?

You have to load headers sequantially to be able to connect them and determine
the longest chain.

Blocks can be loaded in random order once you have their order given by the
headers.
Computing the UTXO however will force you to at least temporarily store the
blocks unless you have plenty of RAM.

Regards,

Tamas Blummer
http://bitsofproof.com

On 07.04.2014, at 21:30, Paul Lyon pml...@hotmail.ca wrote:

I hope I'm not thread-jacking here, apologies if so, but that's the approach
I've taken with the node I'm working on.

Headers can be downloaded and stored in any order, it'll make sense of what
the winning chain is. Blocks don't need to be downloaded in any particular
order and they don't need to be saved to disk, the UTXO is fully
self-contained. That way the concern of storing blocks for seeding (or not)
is wholly separated from syncing the UTXO. This allows me to do the initial
blockchain sync in ~6 hours when I use my SSD. I only need enough disk space
to store the UTXO, and then whatever amount of block data the user would want
to store for the health of the network.

This project is a bitcoin learning exercise for me, so I can only hope I
don't have any critical design flaws in there. :)

From: ta...@bitsofproof.com
Date: Mon, 7 Apr 2014 21:20:31 +0200
To: gmaxw...@gmail.com
CC: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Why are we bleeding nodes?

Once headers are loaded first there is no reason for sequential loading.

Validation has to be sequantial, but that step can be deferred until the
blocks before a point are loaded and continous.

Tamas Blummer
http://bitsofproof.com

On 07.04.2014, at 21:03, Gregory Maxwell gmaxw...@gmail.com wrote:

On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com wrote:
therefore I guess it is more handy to return some bitmap of pruned/full
blocks than ranges.

A bitmap also means high overhead and— if it's used to advertise
non-contiguous blocks— poor locality, since blocks are fetched
sequentially.

--
Put Bad Developers to Shame Dominate Development with Jenkins Continuous
Integration Continuously Automate Build, Test Deployment Start a new
project now. Try Jenkins in the cloud.http://p.sf.net/sfu/13600_Cloudbees
___ Bitcoin-development mailing
list
Bitcoin-development@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bitcoin-development

signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Ricardo Filipe

Or have blocks distributed through pruned nodes as a DHT.

2014-04-07 20:13 GMT+01:00 Mark Friedenbach m...@monetize.io:


 On 04/07/2014 12:20 PM, Tamas Blummer wrote:
 Validation has to be sequantial, but that step can be deferred until the
 blocks before a point are loaded and continous.

 And how do you find those blocks?

 I have a suggestion: have nodes advertise which range of full blocks
 they possess, then you can perform synchronization from the adversed ranges!

 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Tier Nolan

On Mon, Apr 7, 2014 at 8:50 PM, Tamas Blummer ta...@bitsofproof.com wrote:

 You have to load headers sequantially to be able to connect them and
 determine the longest chain.


The isn't strictly true.  If you are connected to a some honest nodes, then
you could download portions of the chain and then connect the various
sub-chains together.

The protocol doesn't support it though.  There is no system to ask for
block headers for the main chain block with a given height,

Finding one high bandwidth peer to download the entire header chain
sequentially is pretty much forced.  The client can switch if there is a
timeout.

Other peers could be used to parallel download the block chain while the
main chain is downloading.  Even if the header download stalled, it
wouldn't be that big a deal.

 Blocks can be loaded in random order once you have their order given by
the headers.
 Computing the UTXO however will force you to at least temporarily store
the blocks unless you have plenty of RAM.

You only need to store the UTXO set, rather than the entire block chain.

It is possible to generate the UTXO set without doing any signature
verification.

A lightweight node could just verify the UTXO set and then do random
signature verifications.

The keeps disk space and CPU reasonably low.  If an illegal transaction is
added to be a block, then proof could be provided for the bad transaction.

The only slightly difficult thing is confirming inflation.  That can be
checked on a block by block basis when downloading the entire block chain.

 Regards,
 Tamas Blummer
 http://bitsofproof.com http://bitsofproof.com

On 07.04.2014, at 21:30, Paul Lyon pml...@hotmail.ca wrote:

I hope I'm not thread-jacking here, apologies if so, but that's the
approach I've taken with the node I'm working on.

Headers can be downloaded and stored in any order, it'll make sense of what
the winning chain is. Blocks don't need to be downloaded in any particular
order and they don't need to be saved to disk, the UTXO is fully
self-contained. That way the concern of storing blocks for seeding (or not)
is wholly separated from syncing the UTXO. This allows me to do the initial
blockchain sync in ~6 hours when I use my SSD. I only need enough disk
space to store the UTXO, and then whatever amount of block data the user
would want to store for the health of the network.

This project is a bitcoin learning exercise for me, so I can only hope I
don't have any critical design flaws in there. :)

--
From: ta...@bitsofproof.com
Date: Mon, 7 Apr 2014 21:20:31 +0200
To: gmaxw...@gmail.com
CC: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Why are we bleeding nodes?


Once headers are loaded first there is no reason for sequential loading.

Validation has to be sequantial, but that step can be deferred until the
blocks before a point are loaded and continous.

Tamas Blummer
http://bitsofproof.com

On 07.04.2014, at 21:03, Gregory Maxwell gmaxw...@gmail.com wrote:

On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com
wrote:

therefore I guess it is more handy to return some bitmap of pruned/full
blocks than ranges.


A bitmap also means high overhead and-- if it's used to advertise
non-contiguous blocks-- poor locality, since blocks are fetched
sequentially.



--
Put Bad Developers to Shame Dominate Development with Jenkins Continuous
Integration Continuously Automate Build, Test  Deployment Start a new
project now. Try Jenkins in the cloud.http://p.sf.net/sfu/13600_Cloudbees

___ Bitcoin-development mailing
list Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development




 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] Why are we bleeding nodes?