Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-05-04 Thread Tier Nolan
On Fri, Apr 11, 2014 at 5:54 PM, Gregory Maxwell gmaxw...@gmail.com wrote:

 For the non-error-coded case I believe nodes
 with random spans of blocks works out asymptotically to the same
 failure rates as random.


If each block is really 512 blocks in sequence, then each slot is more
likely to be hit.  It effectively reduces the number of blocks by the
minimum run lengths.

ECC seemed cooler though.


 (The conversation Peter Todd was referring to was one where I was
 pointing out that with suitable error coding you also get an
 anti-censorship effect where its very difficult to provide part of the
 data without potentially providing all of it)


Interesting too.


 I think in the network we have today and for the foreseeable future we
 can reasonably count on there being a reasonable number of nodes that
 store all the blocks... quite likely not enough to satisfy the
 historical block demand from the network alone, but easily enough to
 supply blocks that have otherwise gone missing.


That's true.  Scaling up the transactions per second increases the chance
of data lost.

With side/tree chains, the odds of data loss in the less important chains
increases (though they are by definition lower value chains)
--
Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.  Get 
unparalleled scalability from the best Selenium testing platform available.
Simple to use. Nothing to install. Get started now for free.
http://p.sf.net/sfu/SauceLabs___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-11 Thread Gregory Maxwell
On Thu, Apr 10, 2014 at 10:30 AM, Tier Nolan tier.no...@gmail.com wrote:
 Error correction is an interesting suggestion.

Though I mentioned it, it was in jest— I think right now it would be
an over-design at least for the basic protocol.  Also, storing
'random' blocks has some locality problems, when verifying blocks you
need to obtain them contiguously, and so we can take advantage of the
locality of reference.  For the non-error-coded case I believe nodes
with random spans of blocks works out asymptotically to the same
failure rates as random.

One thing that I like to point out is that there is absolutely no need
for the entire network to use the same p2p protocol. Diversity here
would be very good.  I think it would be really good for someone to
have an alternative p2p protocol using these techniques even though I
think they aren't yet compelling enough to be table stakes in the
basic protocol.

There are some very helpful things you can do with forward error
correction for faster and more efficient block relaying too:
https://en.bitcoin.it/wiki/User:Gmaxwell/block_network_coding

(The conversation Peter Todd was referring to was one where I was
pointing out that with suitable error coding you also get an
anti-censorship effect where its very difficult to provide part of the
data without potentially providing all of it)

 If there was 1 nodes and each stored 0.1% of the blocks, at random, then
 the odds of a block not being stored is 45 in a million.

I think in the network we have today and for the foreseeable future we
can reasonably count on there being a reasonable number of nodes that
store all the blocks... quite likely not enough to satisfy the
historical block demand from the network alone, but easily enough to
supply blocks that have otherwise gone missing.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
Hi Wladimir,

If the motivation of the SPV wallet is to radically extend functionality, as in 
my case, then the index is specific to the added features and the subset of the 
blockchain that is of interest for the wallet.
As you also point out, adding huge generic purpose indices to core would rather 
discourage people using full nodes due to excess requirements. 

I believe nothing would add more to the core’s popularity as a trusted 
background node to SPV than full validation at lowest possible memory, disk and 
CPU footprint.
Serving headers should be default but storing and serving full blocks 
configurable to ranges, so people can tailor to their bandwith and space 
available.

Tamas Blummer
Bits of Proof

On 09.04.2014, at 21:25, Wladimir laa...@gmail.com wrote:
 
 
 Adding a RPC call for a address - utxo query wouldn't be a big deal. It 
 has been requested before for other purposes as well, all the better if it 
 helps for interaction with Electrum.
 
 Spent history would be involve a much larger index, and it's not likely that 
 will end up in bitcoin
 
 Wladimir
 



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
I tend to agree with slush here - counting the IPs in addr broadcasts often
gives a number like 100,000 vs just 10,000 for actually reachable nodes (or
less). It seems like optimising the NAT tunneling code would help. Starting
by adding more diagnostic stuff to the GUI. STUN support may also help.

The main constraint with home devices is not IMHO their actual power but
rather that a lot of people no longer keep computers switched on all the
time. If you don't do that then spv with bundled Core can't help your
security because the spv wallet would always be syncing from the p2p
network for performance reasons.
On 9 Apr 2014 22:13, slush sl...@centrum.cz wrote:

 I believe there're plenty bitcoind instances running, but they don't have
 configured port forwarding properly.There's uPNP support in bitcoind, but
 it works only on simple setups.

 Maybe there're some not yet considered way how to expose these *existing*
 instances to Internet, to strenghten the network. Maybe just self-test
 indicating the node is not reachable from outside (together with short
 howto like in some torrent clients).

 These days IPv6 is slowly deploying to server environments, but maybe
 there's some simple way how to bundle ipv6 tunnelling into bitcoind so any
 instance will become ipv6-reachable automatically?

 Maybe there're other ideas how to improve current situation without needs
 of reworking the architecture.

 Marek


 On Wed, Apr 9, 2014 at 9:33 PM, Gregory Maxwell gmaxw...@gmail.comwrote:

 On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier justusranv...@gmail.com
 wrote:
  Anyone reading the archives of the list will see about triple the
  number of people independently confirming the resource usage problem
  than they will see denying it, so I'm not particularly worried.

 The list has open membership, there is no particular qualification or
 background required to post here. Optimal use of an information source
 requires critical reading and understanding the limitations of the
 medium. Counting comments is usually not a great way to assess
 technical considerations on an open public forum.  Doubly so because
 those comments were not actually talking about the same thing I am
 talking about.

 Existing implementations are inefficient in many known ways (and, no
 doubt, some unknown ones). This list is about developing protocol and
 implementations including improving their efficiency.  When talking
 about incentives the costs you need to consider are the costs of the
 best realistic option.  As far as I know there is no doubt from anyone
 technically experienced that under the current network rules full
 nodes can be operated with vastly less resources than current
 implementations use, it's just a question of the relatively modest
 implementation improvements.

 When you argue that Bitcoin doesn't have the right incentives (and
 thus something??) I retort that the actual resource _requirements_ are
 for the protocol very low. I gave specific example numbers to enable
 correction or clarification if I've said something wrong or
 controversial. Pointing out that existing implementations are not that
 currently as efficient as the underlying requirements and that some
 large number of users do not like the efficiency of existing
 implementations doesn't tell me anything I disagree with or didn't
 already know. Whats being discussed around here contributes to
 prioritizing improvements over the existing implementations.

 I hope this clarifies something.


 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development




 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Wladimir
On Thu, Apr 10, 2014 at 8:38 AM, Mike Hearn m...@plan99.net wrote:

 I tend to agree with slush here - counting the IPs in addr broadcasts
 often gives a number like 100,000 vs just 10,000 for actually reachable
 nodes (or less). It seems like optimising the NAT tunneling code would
 help. Starting by adding more diagnostic stuff to the GUI. STUN support may
 also help.

 The main constraint with home devices is not IMHO their actual power but
 rather that a lot of people no longer keep computers switched on all the
 time. If you don't do that then spv with bundled Core can't help your
 security because the spv wallet would always be syncing from the p2p
 network for performance reasons.

I agree that there is a fundamental incompatibility in usage between
wallets and nodes. Wallets need to be online as little as possible, nodes
need to online as much as possible.

However, a full node background process could also be running if the wallet
is not open itself. Ffor example - by running as a system service.

Bitcoin Core's own wallet is also moving to SPV, so this means a general
solution is needed to get people to run a node when the wallet is not
running.

Maybe the node shouldn't be controlled from the wallet at all, it could be
a 'node control' user interface on its own (this is what -disablewallet
does currently). In this case, there is no need for packaging it with a
wallet The only drawback would be that initially, people wouldn't know why
or when to install this, hence my suggestion to pack it with wallets...

Wladimir
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn
It's an optimisation problem. Home environments are much more hostile than
servers are due to things like virus scanners, wildly varying memory
pressure as apps are started and shut down, highly asymmetrical upstream
versus downstream bandwidth,  complicated nat setups, people who only use
laptops (which I think is most people these days) and so on.

So I think the right way to go is to optimise the things that hurt server
node operators like large memory and disk  usage, and this will
automatically make it more pleasant to run on the desktop as well. If at
some point all the low hanging fruit for the server side is gone then
improving things on the desktop would be the next place to go. But we have
to be realistic. Desktop tower machines that are always on are dying and
will not be coming back. Not a single person I know uses them anymore, they
have been wiped out in favour of laptops. This is why, given the tiny size
of the bitcoin core development team, I do not think it makes sense to
spend precious coding hours chasing this goal.
On 10 Apr 2014 08:51, Wladimir laa...@gmail.com wrote:

 On Thu, Apr 10, 2014 at 8:38 AM, Mike Hearn m...@plan99.net wrote:

 I tend to agree with slush here - counting the IPs in addr broadcasts
 often gives a number like 100,000 vs just 10,000 for actually reachable
 nodes (or less). It seems like optimising the NAT tunneling code would
 help. Starting by adding more diagnostic stuff to the GUI. STUN support may
 also help.

 The main constraint with home devices is not IMHO their actual power but
 rather that a lot of people no longer keep computers switched on all the
 time. If you don't do that then spv with bundled Core can't help your
 security because the spv wallet would always be syncing from the p2p
 network for performance reasons.

 I agree that there is a fundamental incompatibility in usage between
 wallets and nodes. Wallets need to be online as little as possible, nodes
 need to online as much as possible.

 However, a full node background process could also be running if the
 wallet is not open itself. Ffor example - by running as a system service.

 Bitcoin Core's own wallet is also moving to SPV, so this means a general
 solution is needed to get people to run a node when the wallet is not
 running.

 Maybe the node shouldn't be controlled from the wallet at all, it could be
 a 'node control' user interface on its own (this is what -disablewallet
 does currently). In this case, there is no need for packaging it with a
 wallet The only drawback would be that initially, people wouldn't know why
 or when to install this, hence my suggestion to pack it with wallets...

 Wladimir


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
You ask why people would install this ?

I find it is odd that we who hold the key to instant machine to machine micro 
payments do not use it to incentivise committing resources to the network.
What about serving archive blocks to peers paying for it ?

Tamas Blummer
http://bitsofproof.com

On 10.04.2014, at 08:50, Wladimir laa...@gmail.com wrote:
 The only drawback would be that initially, people wouldn't know why or when 
 to install this, hence my suggestion to pack it with wallets...
 
 Wladimir
 



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn

 I find it is odd that we who hold the key to instant machine to machine
 micro payments do not use it to incentivise committing resources to the
 network.


It's not a new idea, obviously, but there are some practical consequences:

1) To pay a node for serving, you have to have bitcoins. To get bitcoins,
you need to sync with the network via a node. Catch 22.

2) If some nodes choose to charge and others choose to not charge, a smart
wallet will always use the free nodes. In the absence of any global load
balancing algorithms, this would lead to the free nodes getting overloaded
and collapsing whilst the for-pay nodes remain silent.

3) The only payment channel implementations today are bitcoinj's (Java) and
one written by Jeff in Javascript. There are no C++ implementations. And as
Matt and I can attest to, doing a real, solid, fully debugged
implementation that's integrated into a real app is  a lot of work.

I still think the lowest hanging fruit is basic, boring optimisations
rather than architectural rethinks.
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



 But we
have
to be realistic. Desktop tower machines that are always on are dying
and
will not be coming back. Not a single person I know uses them anymore,
they
have been wiped out in favour of laptops. This is why, given the tiny
size
of the bitcoin core development team, I do not think it makes sense to
spend precious coding hours chasing this goal.

Your social group is weird.

Nearly every coworker at my previous job had a tower computer at work and at 
home. Equally in my nontechnical social group lots of people, a significant 
minority if not majority, have Apple and PC desktops hooked up to large 
monitors at home for media production and games. Those who don't more often 
than not have laptops used as desktops, sitting in one place 95% of the time 
and left on.

People have found it most efficient to work at a static desk for centuries - 
that's not going away. Of course we're seeing desktop usage and sales falling, 
but that's only because previously the mobile usage was forced into suboptimal 
options by technical realities. The trend will bottom out a long way from zero.

Besides, even if just 1% of bitcoin users had a machine they left on that could 
usefully contribute to the network it would still vastly outweigh the much 
smaller percentage who would run nodes on expensive hosted capacity out of the 
goodness of their hearts. If we educated users about the privacy advantages of 
full nodes and gave them software that automatically contributed back within 
defined limits we'd have tens of thousands more useful nodes in the exact same 
way that user friendly filesharing software has lead to millions of users 
contributing bandwidth to filesharing networks. Similarly take advantage of the 
fault tolerance inherent in what we're doing and ensure that our software can 
shrug off nodes with a few % of downtime - certainly possible.

Of course, this doesn't fit in the business plans of those who might want to 
run full nodes to data mine and deanonymize users for marketing, tax 
collection, and law enforcement - one of the few profitable things to do with a 
full node - but screw those people.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCgA6BQJTRmVPMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhYb9B/0QWgQusFrQoPfYIzig
vpeo9sQgHAfA3gPYdqDLTtlaTgt8w3oP+/N46+Pi3lEphhCusXstzAFxi6c0XXsk
s96l9MqrUCZu55oEj1wZP0LJQx1uyUdevFv9bmocG5+94jBvGIoy3JZ3iQg+nNjL
uE9FpTnL43VOZ0WI9g6pXIE6XX6CxTx56tcxla4LTTypA1DijXa9MyYGOfYxXdPJ
w7jiRKl6Ijb3twP8+pX07GSIlL9yP7bESydnwyzwEo/RxAJxPmUpLuxluQ5DKTNY
G9TtwCpT+c6g5nXOxkI31XRcDuzhT+2kEhiDA6Neu2YNGrnyQx2WL6XuZNhi8nKB
IOMm
=rwGV
-END PGP SIGNATURE-


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
I know the idea is not new. Just bringing it up to emphasize that if we don’t 
use it how could we expect other networks using it.
Machine to machine micro payments could become the killer application for 
Bitcoin.

1) There is no catch 22 as there are plenty of ways getting bitcoin without 
bootstrapping a full node.

2) let markets work out and not speculate what would happen.

3) Serving archive bolcks does not have to be part of core but could be a 
distinct service written in a language of your choice using new protocol.

As mentioned earlier I am for a stripped down core that does nothing else than 
consensus and stores nothing else needed for that task and offering SPV api to 
the wallets.

Tamas Blummer
http://bitsofproof.com

On 10.04.2014, at 11:17, Mike Hearn m...@plan99.net wrote:

 I find it is odd that we who hold the key to instant machine to machine micro 
 payments do not use it to incentivise committing resources to the network.
 
 It's not a new idea, obviously, but there are some practical consequences:
 
 1) To pay a node for serving, you have to have bitcoins. To get bitcoins, you 
 need to sync with the network via a node. Catch 22.
 
 2) If some nodes choose to charge and others choose to not charge, a smart 
 wallet will always use the free nodes. In the absence of any global load 
 balancing algorithms, this would lead to the free nodes getting overloaded 
 and collapsing whilst the for-pay nodes remain silent.
 
 3) The only payment channel implementations today are bitcoinj's (Java) and 
 one written by Jeff in Javascript. There are no C++ implementations. And as 
 Matt and I can attest to, doing a real, solid, fully debugged implementation 
 that's integrated into a real app is  a lot of work.
 
 I still think the lowest hanging fruit is basic, boring optimisations rather 
 than architectural rethinks.



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 10 April 2014 05:17:28 GMT-04:00, Mike Hearn m...@plan99.net wrote:

 I find it is odd that we who hold the key to instant machine to
machine
 micro payments do not use it to incentivise committing resources to
the
 network.


It's not a new idea, obviously, but there are some practical
consequences:

You're both missing a more important issue: a core security assumption of 
bitcoin is that information is so easy to spread that censorship of it becomes 
impractical. If we're at the point where nodes are charging for their data 
we've failed that assumption.

More concretely, if my business is charging for block chain data and I can make 
a profit doing so via micro payments I have perverse incentives to drive away 
my competitors. If I give a peer a whole block they can sell access to that 
information in turn. Why would I make it easy for them if I don't have too?

Anyway, much of this discussion seems to stem from the misconception that 
contributing back to the network is a binary all or nothing thing - it's not. 
Over a year ago I myself was lamenting how I and the other bitcoin-wizards 
working on scalability had quickly solved every scaling problem *but* how to 
make it possible to scale up and keep mining decentralised.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCgA6BQJTRmi9MxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhT/BB/98DudTV85hmruk0WRS
xVt7gGK6DJ2Isc7iJE09i9wSQc/PbHu7rZ2MYPreOdVzRmlHYhOV2ShnZaZJ7A9g
qB8pwy1wVrZgbrFeXXOLegNxGF2Xzc8OtL1E0bkNtTUUkuPIvT3UV4xn/Z+UZToR
XImXpfakfJvyRH80cbMNu4xG/t7Ym4K63nEpCCdsEKNm5a1vHTNRTNfGYMC1wrSV
XI3boZk7BQjqFDZADnonUU9zQ1WOmpdaVBYm+Xtgc+HXl3QODLcwGCY9hIRvaxek
L+IYX9yTVbgngDGy70BYG4ekWxvMtNzRU+9HG5DgA1/Er9r1KIaf/98xiFR7RVB4
Yfia
=34aD
-END PGP SIGNATURE-


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn

 1) There is no catch 22 as there are plenty of ways getting bitcoin
 without bootstrapping a full node.


I think I maybe wasn't clear. To spend coins you need transaction data.
Today, the dominant model is that people get that data by scanning the
block chain. If you can obtain the transaction data without doing that
then, either:

1) Someone is doing chain scanning for free. See my point about why pay if
you can get it for free.

2) You got your tx data direct from the person you who sent you the funds,
perhaps via the payment protocol. This would resolve the catch 22 by
allowing you to spend bitcoins without actually having talked to the P2P
network first, but we're a loong way from this world.

And that's it. I don't think there are any other ways to get the tx data
you need. Either someone gives it to you in the act of spending, or someone
else gives it away for free, undermining the charge-for-the-p2p-network
model.
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tamas Blummer
Thanks, Peter and you convinced me. I run away with a thought.

It’d be great to find a spot to deploy payment channels, but I agree this is 
not it.

Tamas Blummer
http://bitsofproof.com

On 10.04.2014, at 12:40, Mike Hearn m...@plan99.net wrote:

 1) There is no catch 22 as there are plenty of ways getting bitcoin without 
 bootstrapping a full node.
 
 I think I maybe wasn't clear. To spend coins you need transaction data. 
 Today, the dominant model is that people get that data by scanning the block 
 chain. If you can obtain the transaction data without doing that then, either:
 
 1) Someone is doing chain scanning for free. See my point about why pay if 
 you can get it for free.
 
 2) You got your tx data direct from the person you who sent you the funds, 
 perhaps via the payment protocol. This would resolve the catch 22 by allowing 
 you to spend bitcoins without actually having talked to the P2P network 
 first, but we're a loong way from this world.
 
 And that's it. I don't think there are any other ways to get the tx data you 
 need. Either someone gives it to you in the act of spending, or someone else 
 gives it away for free, undermining the charge-for-the-p2p-network model.



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Wladimir
On Thu, Apr 10, 2014 at 8:04 AM, Tamas Blummer ta...@bitsofproof.comwrote:

 Serving headers should be default but storing and serving full blocks
 configurable to ranges, so people can tailor to their bandwith and space
 available.


I do agree that it is important.

This does require changes to the P2P protocol, as currently there is no way
for a node to signal that they store only part of the block chain. Also,
clients will have to be modified to take this into account. Right now they
are under the assumption that every full node can send them every
(previous) block.

What would this involve?

Do you know of any previous work towards this?

Wladimir
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn

 What would this involve?

 Do you know of any previous work towards this?


Chain pruning is a fairly complicated project, partly because it spans
codebases. For instance if you try and implement it *just* by changing
Bitcoin Core, you will break all the SPV clients based on bitcoinj (i.e.
all of them). Big changes to the P2P network like this require upgrading
both codebases simultaneously.

I think things like this may be why Gavin is now just chief scientist
instead of Core maintainer - in future, the changes people need will span
projects and require fairly significant planning.

From a technical perspective, it means extending addr broadcasts so nodes
broadcast how much of the chain they have, and teaching both Core and
bitcoinj how to search for nodes that have enough of the chain for them to
use. Currently bitcoinj still doesn't use addr broadcasts at all, there's
an incomplete patch available but it was never finished or merged. So that
has to be fixed first. And that probably implies improving Bitcoin Core so
the results of getaddr are more usable, ideally as high quality as what the
DNS seeds provide, because if lots of bad addresses are returned this will
slow down initial connect time, which is an important performance metric.
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Pieter Wuille
There were earlier discussions.

The two ideas were either using one or a few service bits to indicate
availability of blocks, or to extend addr messages with some flags to
indicate this information.

I wonder whether we can't have a hybrid: bits to indicate general
degree of availability of blocks (none, only recent, everything), but
indicate actual availability only upon actually connecting (through a
version extension, or - preferably - a separate message). Reason is
that the actual blocks available are likely to change frequently (if
you keep the last week of blocks, a 3-day old addr entry will have
quite outdated information), and not that important to actual peer
selection - only to drive the decision which blocks to ask after
connection.

On Thu, Apr 10, 2014 at 1:09 PM, Wladimir laa...@gmail.com wrote:
 On Thu, Apr 10, 2014 at 8:04 AM, Tamas Blummer ta...@bitsofproof.com
 wrote:

 Serving headers should be default but storing and serving full blocks
 configurable to ranges, so people can tailor to their bandwith and space
 available.


 I do agree that it is important.

 This does require changes to the P2P protocol, as currently there is no way
 for a node to signal that they store only part of the block chain. Also,
 clients will have to be modified to take this into account. Right now they
 are under the assumption that every full node can send them every (previous)
 block.

 What would this involve?

 Do you know of any previous work towards this?

 Wladimir


 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 10 April 2014 06:44:32 GMT-04:00, Tamas Blummer ta...@bitsofproof.com 
wrote:
Thanks, Peter and you convinced me. I run away with a thought.

It’d be great to find a spot to deploy payment channels, but I agree
this is not it.

No problem!

I'm sure we'll see payment channels implemented sooner or later
the form of hub and spoke payment networks. The idea there is you have one or 
more centralised hubs who in turn have payment channels setup to and from 
payors and payees. So long as the person you want to pay is connected to the 
same hub as you are, or in more advanced versions connected via a ripple style 
chain, you can push payment to the hub and get proof they did the same for the 
recipient. Your loss is always limited to the incremental payment amount and 
payment is essentially instant.

Of course, it's got some disadvantages compared to standard bitcoin 
transactions - its less decentralised - but when compared to other forms of 
off-chain payment in most situations its a strict improvement, and having the 
capability available is always a strict improvement. Like fidelity bonded banks 
the trust required in the hubs is low enough that with some minor effort 
applied to anti-DoS you could probably get away with using even hubs run by 
anonymous actors, making the centralisation less important. (hubs are 
essentially interchangeable) Unlike pure fidelity bonded banks the effort 
required to build this is relatively minor!

You can even combine it with chaum tokens for anonymity. You'll want to hold 
the tokens for some amount of time to thwart timing analysis, leaving you 
somewhat vulnerable to theft, but in that case fidelity bonded banking 
principles can be applied. Other than that case the idea is probably made 
obsolete by micropayment hubs.

Regulatory issues will be interesting... If you wind up with a few central 
payment hubs, without chaum tokens, those hubs learn all details about every 
transaction made. Obviously if a big actor like BitPay implemented this there 
would be a lot of pressure on them to make those records available to law 
enforcement and tax authorities, not to mention marketing and other data 
mining. Equally I suspect that if an alternative more decentralised version was 
implemented there would be strong government pressure for those approved hubs 
to not interoperate with the decentralised hubs, and equally for merchants to 
not accept payment from the decentralised hubs.

But all the same, if widely implemented this reduces pressure to raise the 
block size enormously, keeping the underlying system decentralised. So the net 
effect is probably positive regardless.

Oh yeah, credit goes to Mike Hearn for the payment channels, and if I'm 
correct, for the hub concept as well.

Amir: You should think about adding the above to dark wallet. It'd be good if 
the protocols are implemented in an open and decentralised fashion first, prior 
to vendor lock in.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCgA6BQJTRoIlMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhW26B/9A9OtYjoSHo620XZzF
VqfnnVFCPr3DpD/HuT3JYhF1kkL2vTt5wkRIHmHmfJ29Sduj8St7EFiLOyUg2mvt
q9heZgzCnqxLJm9zMiiQnb3Y/plvhTLfaONnHI+OPSfrABL6DA04nEe8OBPuFfv/
NowJ74DP/65YBq3EqbqG0dJExKm1BhdrEpWNq0v5YoCVuEYkWgFHL8SdRHnfFyxA
XTkP8avzlG82r98k55IrV0O/6VQNHjE0+xF4EHjEYBacy6OwlpEYeLrqx/VDAQ5R
RufXeAltNZI0tzLQ8nY0aMBH3YFxF0+14/sbmOAtmnD6EW49gEcV9MnSJc5ct4m7
Szq5
=aC39
-END PGP SIGNATURE-


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 10 April 2014 07:32:44 GMT-04:00, Pieter Wuille pieter.wui...@gmail.com 
wrote:
There were earlier discussions.

The two ideas were either using one or a few service bits to indicate
availability of blocks, or to extend addr messages with some flags to
indicate this information.

I wonder whether we can't have a hybrid: bits to indicate general
degree of availability of blocks (none, only recent, everything), but
indicate actual availability only upon actually connecting (through a
version extension, or - preferably - a separate message). Reason is
that the actual blocks available are likely to change frequently (if
you keep the last week of blocks, a 3-day old addr entry will have
quite outdated information), and not that important to actual peer
selection - only to drive the decision which blocks to ask after
connection.

Why not just put an expiration date on that information and delay deletion 
until the expiration is reached?

Also, its worth noting how the node bit solution you proposed can be done as a 
gradual upgrade path for SPV client. From the perspective of nodes that don't 
know about it they just see the pruned nodes as SPV nodes without any chain 
data at all. The only issue would be if large numbers of uses turned off their 
full nodes, but that's a possibility regardless. Done with partial UTXO set 
mode this may even result in an eventual increase in the number of full nodes.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCgA6BQJTRoPZMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhetrCACA02EJQ0VpcYvafuNc
7pvqMVeirJRu3Uv7Wy8rcl9jW5irM5fmNdznARtv2vwpEZN7MU0wp3ZY1FYOCv2f
PvWC7DBCSBs2BuyGkvPuwnXEppTrYmWFT3qjg+99lF1IlOV4yWFacja2RGDuJkea
fYUkODosHJjFVcXi5aMkBPQ5sOFdlUVbC94YV4d4PDSmF2fHLGG8uEfEweYb6Pv+
gj1CsfuAWf8DWzygDeL8x/wOG9HeqYqEbjxyOb9hxlp1ByUof+4WJtz3QfGsR2Xt
fvkmgS8vkUxSIZorMdypj7oLBOnfDW1bEK5He2SlqPdYi5FEQusZ/jMMX3Fw74GV
fJKt
=Wyv8
-END PGP SIGNATURE-


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Mike Hearn

 Oh yeah, credit goes to Mike Hearn for the payment channels, and if I'm
 correct, for the hub concept as well.


Actually, the design is from Satoshi and Matt did most of the
implementation work last year during a Google internship. Though I ended up
doing a lot of work on it too. We actually got pretty far: there was
Android UI for it and a couple of apps we coded up. I wish we could have
pushed it over the finishing line and got real world usage. Hopefully we
can return to it someday soon.

I think the hub/spoke concept was invented by goldsmiths in 16th century
Italy, as they started handing pieces of paper across their benches, or
*bancos* in Italian   :-)
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Gregory Maxwell
On Thu, Apr 10, 2014 at 4:32 AM, Pieter Wuille pieter.wui...@gmail.com wrote:
 There were earlier discussions.

On this list.

 The two ideas were either using one or a few service bits to indicate
 availability of blocks, or to extend addr messages with some flags to
 indicate this information.

 I wonder whether we can't have a hybrid: bits to indicate general
 degree of availability of blocks (none, only recent, everything), but
 indicate actual availability only upon actually connecting (through a
 version extension, or - preferably - a separate message). Reason is
 that the actual blocks available are likely to change frequently (if
 you keep the last week of blocks, a 3-day old addr entry will have
 quite outdated information), and not that important to actual peer
 selection - only to drive the decision which blocks to ask after
 connection.

I think you actually do need the kept ranges to be circulated,
otherwise you might need to hunt for a very long time to find the
right nodes with the blocks you need.  Alternatively, you give up and
don't hunt and pick some node that has them all and we get poor load
distribution. I'd rather be in a case where the nodes that have the
full history are only hit as a last resort.

WRT the changing values, I think that is pretty uniquely related to
the most recent blocks, and so instead I think that should be handled
symbolically (e.g. the hybrid approach... a flag for the I keep the
most recent 2000 blocks, I say 2000 because thats about where the
request offset historgrams flattened out) or as a single offset range
I keep the last N=200,  and the flag or the offset would be in
addition to whatever additional range was signaled. The latter could
be infrequently changing.

Signaling _more_ and more current range data on connect seems fine to
me, I just don't think it replaces something that gets relayed.

Based on the safety against reorgs and the block request access
patterns we observed I'm pretty sure we'd want any node serving blocks
at all to be at least the last N (for some number between 144 and 2000
or so). Based on the request patterns if just the recent blocks use up
all the space you're willing to spend, then I think thats probably
still the optimal contribution...

(Just be glad I'm not suggesting coding the entire blockchain with an
error correcting code so that it doesn't matter which subset you're
holding)

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 10 April 2014 07:45:16 GMT-04:00, Mike Hearn m...@plan99.net wrote:

 Oh yeah, credit goes to Mike Hearn for the payment channels, and if
I'm
 correct, for the hub concept as well.


Actually, the design is from Satoshi and Matt did most of the
implementation work last year during a Google internship.

Ah right, of course. Along those lines we should credit Jeremy Spilman (?) for 
figuring out how to get rid of the dependency on nSequence, makimg the protocol 
trust-free.

I do recall it having an issue with malleability, semi-fixed with the P2SH 
trick. Be good to clear that up for good for Pieter's proposed malleability 
patch.

Though I
ended up
doing a lot of work on it too. We actually got pretty far: there was
Android UI for it and a couple of apps we coded up. I wish we could
have
pushed it over the finishing line and got real world usage. Hopefully
we
can return to it someday soon.

I think the hub/spoke concept was invented by goldsmiths in 16th
century
Italy, as they started handing pieces of paper across their benches, or
*bancos* in Italian   :-)

...and only took another five hundred years for math to catch up and make it 
trust free, modulo miner centralisation!
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCgA6BQJTRoYCMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhdNMB/9zFBT3nFSkasHkMVW8
01pE/VYrclH5BsxANakOqfb57Rprc4xue9H4AuppqAlIo/i/gYtOyoDy9y8oQDmC
YRiM6K5+bqUH8uC/Rjz1AASBrnb3zAasqCJlklheb5nP3+EoLpKNJ04Brk8rnlqp
CMO5GktE43r9buYL9MOMXHanB523wRmNV8JjpEF+y5tPYe9YW7rsdmpjX8F8sRga
PE1MMNy43lctoej4tR0iSBfK2ZNudsz7PdzW9+4Gvpc4NDMHp5O4JwhX/vPbSyyC
+Or1BvPKe58zVSi5kBi7AJEeFXasfuKpwBwT9r2CGmjvI62ESmkj5M9eqcb4i8Yy
i2Zr
=E4NH
-END PGP SIGNATURE-


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 10 April 2014 07:50:55 GMT-04:00, Gregory Maxwell gmaxw...@gmail.com wrote:
(Just be glad I'm not suggesting coding the entire blockchain with an
error correcting code so that it doesn't matter which subset you're
holding)

I forgot to ask last night: if you do that, can you add new blocks to the chain 
with the encoding incrementally?
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCgA6BQJTRoZ+MxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhYudCAC7ImifMnLIFHv1UifV
zRxtDkx7UxIf9dncDAcrTIyKEDhoouh0TmoZl3HKQ3KUEETAVKsMzqXLgqVe6Ezr
ny1bm0pQlkBCZFRwuZvmB27Y3mwC8PD6rT9ywtWzFjWd8PEg6/UaM547nQPw7ir0
27S3XMfE/BMiQWfWnWc/nqpbmJjd8x/dM3oiTG9SVZ7iNxotxAqfnW2X5tkhJb0q
dAV08wpu6aZ5hTyLpvDxXDFjEG119HJeLkT9QVIrg+GBG55PYORqE4gQr6uhrF4L
fGZS2EIlbk+kAiv0EjglQfxWM7KSRegplSASiKEOuX80tqLIsEugNh1em8qvG401
NOAS
=CWql
-END PGP SIGNATURE-


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-10 Thread Tier Nolan
Error correction is an interesting suggestion.

If there was 1 nodes and each stored 0.1% of the blocks, at random,
then the odds of a block not being stored is 45 in a million.

Blocks are stored on average 10 times, so there is already reasonable
redundancy.

With 1 million blocks, 45 would be lost in that case, even though most are
stored multiple times.

With error correction codes, the chances of blocks going missing is much
lower.

For example, if there was 32 out of 34 Reed-Solomon-like system, then 2
blocks out of 34 could be lost without any actual data loss for the network.

As a back of the envelop check, the odds of 2 missing blocks landing within
34 of another is 68/100.  That means that the odds of 2 missing blocks
falling in the same correction section is 45 * 34 / 100 = 0.153%.  Even
in that case, the missing blocks could be reconstructed, as long as you
know that they are missing.

The error correction code has taken it from being a near certainty that
some blocks would be lost to less than 0.153%.

A simple error correction system would just take 32 blocks in sequence and
then compute 2 extra blocks.

The extra blocks would have to be the same length as the longest block in
the 32 being corrected.

The shorter blocks would be padded with zeroes so everything is the same
size.

For each byte position in the blocks you compute the polynomial that goes
through byte (x, data(x)), for x = 0 to 31.  This could be a finite field,
or just mod 257.

You can then compute the value for x=32 and x = 33.  Those are the values
for the 2 extra blocks.

If mod 257 is used, then only the 2 extra blocks have to deal with symbols
from 0 to 256.

If you have 32 of the 34 blocks, you can compute the polynomial and thus
generate the 32 actual blocks.

This could be achieved by a soft fork by having a commitment every 32
blocks in the coinbase.

It makes the header chain much longer though.

Longer sections are more efficient, but need more calculations to recover
everything.  You could also do interleaving to handle the case where entire
sections are missing.


On Thu, Apr 10, 2014 at 12:54 PM, Peter Todd p...@petertodd.org wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA512



 On 10 April 2014 07:50:55 GMT-04:00, Gregory Maxwell gmaxw...@gmail.com
 wrote:
 (Just be glad I'm not suggesting coding the entire blockchain with an
 error correcting code so that it doesn't matter which subset you're
 holding)

 I forgot to ask last night: if you do that, can you add new blocks to the
 chain with the encoding incrementally?
 -BEGIN PGP SIGNATURE-
 Version: APG v1.1.1

 iQFQBAEBCgA6BQJTRoZ+MxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhYudCAC7ImifMnLIFHv1UifV
 zRxtDkx7UxIf9dncDAcrTIyKEDhoouh0TmoZl3HKQ3KUEETAVKsMzqXLgqVe6Ezr
 ny1bm0pQlkBCZFRwuZvmB27Y3mwC8PD6rT9ywtWzFjWd8PEg6/UaM547nQPw7ir0
 27S3XMfE/BMiQWfWnWc/nqpbmJjd8x/dM3oiTG9SVZ7iNxotxAqfnW2X5tkhJb0q
 dAV08wpu6aZ5hTyLpvDxXDFjEG119HJeLkT9QVIrg+GBG55PYORqE4gQr6uhrF4L
 fGZS2EIlbk+kAiv0EjglQfxWM7KSRegplSASiKEOuX80tqLIsEugNh1em8qvG401
 NOAS
 =CWql
 -END PGP SIGNATURE-



 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
YES

Such a bitcoind is what I called border router in a previous mail. 

Yes, SPV wallets are getting ahead of features, so people will use them also 
because on size just does not fit all, but all want to ensure being on the same 
trunk of the chain.
Therefore serious user of Bitcoin run a bitcoind as a border router and connect 
SPV wallets with higher functionality to that trusted node(s).

This is what I think the core should focus on: Being a lightweight superfast 
consensus building border router and nothing more. No wallet, no GUI, no RPC 
calls,
no Payment protocol and the rest.

Regards,

Tamas Blummer
http://bitsofproof.com

On 09.04.2014, at 17:29, Wladimir laa...@gmail.com wrote:

 Hello,
 
 This is primarily aimed at developers of SPV wallets.
 
 The recently reported decrease in number of full nodes could have several 
 reasons, one of them that less people are running Bitcoin Core for the wallet 
 because the other wallets are getting ahead in both features and useability.
 
 It's great to see innovation in wallets, but it's worrying that the number of 
 full nodes decreases. 
 
 It may be that lots of people would support the network by running a full 
 node, but don't want to go through the trouble of installing bitcoin core 
 separately (and get confused because it's a wallet, too).
 
 Hence I'd like to explore the idea of adding an option to popular SPV 
 wallets, to spin a bitcoind process in the background. This could be pretty 
 much transparent to the user - it would sync in the background, the wallet 
 could show statistics about the node, but is not dependent on it.
 
 In exchange the user would get increased (full node level) security, as the 
 SPV wallet would have a local trusted node.
 
 Does this sound like a good idea?
 
 Is there any way that Bitcoin Core can help to accomedate this 'embedded' 
 usage? Specific Interfaces, special builds - maybe add a walletless bitcoind 
 build to gitian - bindings, dlls, etc?
 
 Wladimir
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Brian Hoffman
How would this affect the user in terms of disk storage? They're going to
get hammered on space constraints aren't they? If it's not required how
likely are users to enable this?


On Wed, Apr 9, 2014 at 11:29 AM, Wladimir laa...@gmail.com wrote:

 Hello,

 This is primarily aimed at developers of SPV wallets.

 The recently reported decrease in number of full nodes could have several
 reasons, one of them that less people are running Bitcoin Core for the
 wallet because the other wallets are getting ahead in both features and
 useability.

 It's great to see innovation in wallets, but it's worrying that the number
 of full nodes decreases.

 It may be that lots of people would support the network by running a full
 node, but don't want to go through the trouble of installing bitcoin core
 separately (and get confused because it's a wallet, too).

 Hence I'd like to explore the idea of adding an option to popular SPV
 wallets, to spin a bitcoind process in the background. This could be pretty
 much transparent to the user - it would sync in the background, the wallet
 could show statistics about the node, but is not dependent on it.

 In exchange the user would get increased (full node level) security, as
 the SPV wallet would have a local trusted node.

 Does this sound like a good idea?

 Is there any way that Bitcoin Core can help to accomedate this 'embedded'
 usage? Specific Interfaces, special builds - maybe add a walletless
 bitcoind build to gitian - bindings, dlls, etc?

 Wladimir



 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 8:42 AM, Brian Hoffman brianchoff...@gmail.com wrote:
 How would this affect the user in terms of disk storage? They're going to
 get hammered on space constraints aren't they? If it's not required how
 likely are users to enable this?

If Bitcoin core activates pruning a full node can be supported in—
say— 4GBytes or so. (That gives enough space to store the utxo about
350MB now, and a couple gigs for blocks to serve out).

I'd imagine getting information from SPV wallet developers how much
disk usage agility they think is required is part of what Wladimir is
looking for.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 8:41 AM, Natanael natanae...@gmail.com wrote:
 This could probably be done fairly easily by bundling Stratum (it's
 not just for pools!) and allowing SPV wallets to ask Bitcoind to start
 it (if you don't use it, there's no need to waste the resources), and
 then connect to it. The point of using Stratum is that it already is
 being used by Electrum,

Sadly today Electrum requires more than a full node, it requires a
number of large additional indexes over what a full node has and
pruning is precluded. I don't think that increasing the resource
utilization of the node is a good way to go there for the purposes
expressed here. (not that electrum couldn't be used here, but not
unmodified without the resource usage increasing route)

 and that it might be an easier way to support
 SPV clients than creating a new API in bitcoind for it since Stratum
 itself already relies on bitcoind to provide it's services.

Bitcoin's own P2P protocol is already the API for a ordinary SPV
client. So I don't believe any new API would be require, except
perhaps for some process management stuff (which also isn't provided
for Electrum).

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Thomas Voegtlin
Le 09/04/2014 17:54, Gregory Maxwell a écrit :

 Sadly today Electrum requires more than a full node, it requires a
 number of large additional indexes over what a full node has and
 pruning is precluded. I don't think that increasing the resource
 utilization of the node is a good way to go there for the purposes
 expressed here. (not that electrum couldn't be used here, but not
 unmodified without the resource usage increasing route)


Electrum uses two large indexes:

 address - utxo

(patricia tree, aka ultimate blockchain compression, see thread 
started by Alan Reiner in the bitcointalk forum)

 address - spent history

The first index is not going to grow larger than what bitcoind already 
needs to store, because bitcoind will always need to store utxos.

The second index threatens to become large. However, Electrum servers do 
not keep the full histories, they prune older entries. Without adapting 
Electrum clients, it would even be possible to keep only one bit per 
address (to know whether that address has been used or not), and that 
information is only used to restore wallets from seed, not during normal 
operations.

If the first index (patricia tree) was implemented in bitcoind, that 
would obviously be a big relief for electrum servers.



 and that it might be an easier way to support
 SPV clients than creating a new API in bitcoind for it since Stratum
 itself already relies on bitcoind to provide it's services.

 Bitcoin's own P2P protocol is already the API for a ordinary SPV
 client. So I don't believe any new API would be require, except
 perhaps for some process management stuff (which also isn't provided
 for Electrum).

 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development



--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
I am glad that SPV wallets are discussed outside the scope of mobile devices!

Yes, SPV is a sufficient API to a trusted node to build sophisticated features 
not offered by the core.
SPV clients of the border router will build their own archive and indices based 
on their interest of the chain therefore the
border router core does not need to store (and process) anything not needed for 
consensus, its memory
or disk footprint would be as low as an optimal storage of UTXO.

Regards,

Tamás Blummer
http://bitsofproof.com

On 09.04.2014, at 17:57, Gregory Maxwell gmaxw...@gmail.com wrote:

 On Wed, Apr 9, 2014 at 8:42 AM, Brian Hoffman brianchoff...@gmail.com wrote:
 How would this affect the user in terms of disk storage? They're going to
 get hammered on space constraints aren't they? If it's not required how
 likely are users to enable this?
 
 If Bitcoin core activates pruning a full node can be supported in—
 say— 4GBytes or so. (That gives enough space to store the utxo about
 350MB now, and a couple gigs for blocks to serve out).
 
 I'd imagine getting information from SPV wallet developers how much
 disk usage agility they think is required is part of what Wladimir is
 looking for.
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment 
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mark Friedenbach
On 04/09/2014 09:09 AM, Tamas Blummer wrote:
 Yes, SPV is a sufficient API to a trusted node to build sophisticated
 features not offered by the core.
 SPV clients of the border router will build their own archive and
 indices based on their interest of the chain therefore the
 border router core does not need to store (and process) anything not
 needed for consensus, its memory
 or disk footprint would be as low as an optimal storage of UTXO.

Storing zero full blocks does nothing to aid the network.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 5:41 PM, Natanael natanae...@gmail.com wrote:

 This could probably be done fairly easily by bundling Stratum (it's
 not just for pools!) and allowing SPV wallets to ask Bitcoind to start
 it (if you don't use it, there's no need to waste the resources), and
 then connect to it. The point of using Stratum is that it already is
 being used by Electrum, and that it might be an easier way to support
 SPV clients than creating a new API in bitcoind for it since Stratum
 itself already relies on bitcoind to provide it's services.


Why would a new API be needed? (beside maybe some functionality that would
make it easier to integrate)

P2P should be enough for SPV clients such as BitcoinJ to get access to
(filtered) blocks and transations, and RPC can be used to manage/query the
bitcoind instance. I'm not sure what stratum would add.

Wladimir
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Alex Mizrahi

 1) It's more private. Bloom filters gives away quite accurate statistical
 information about what coins you own to whom ever you happen to be
 connected too. An attacker can easily use this to deanonymize you even if
 you don't reuse addresses; Tor does not help much against this attack.


There is also an option to download everything, but do only a very basic
surface validation (without keeping track of UTXOs).
You do not need a full node for that.
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 7:33 PM, Alex Mizrahi alex.mizr...@gmail.com wrote:

 1) It's more private. Bloom filters gives away quite accurate statistical
 information about what coins you own to whom ever you happen to be
 connected too. An attacker can easily use this to deanonymize you even if
 you don't reuse addresses; Tor does not help much against this attack.


 There is also an option to download everything, but do only a very basic
 surface validation (without keeping track of UTXOs).
 You do not need a full node for that.


You may not *need* a full node, but the point of this (which I clearly
explained in my opening post) would be to support the network.

Wladimir
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 9 April 2014 13:33:19 GMT-04:00, Alex Mizrahi alex.mizr...@gmail.com wrote:

 1) It's more private. Bloom filters gives away quite accurate
statistical
 information about what coins you own to whom ever you happen to be
 connected too. An attacker can easily use this to deanonymize you
even if
 you don't reuse addresses; Tor does not help much against this
attack.


There is also an option to download everything, but do only a very
basic
surface validation (without keeping track of UTXOs).
You do not need a full node for that.

Yup, that's part of the idea behind partial UTXO set mode. You could have a 
model where your node starts with no data at all, and hence SPV security. You 
tell your node what the oldest key birthday is that your interested in and it 
downloads the full block chain starting at that date, giving you your txs w/ 
SPV security and full node privacy.

What partial UTXO would add on top of that is then your node would gradually 
scan backwards until block zero, at which point it has a complete UTXO set and 
is a full node.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCgA6BQJTRYWUMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhaHEB/0Q0P544nJrtapnzogG
QpREeBlwR5I8fZ+BCbNcAk6afk9KLOFIsLBW0bFvFXfSO1TiyCLMezLkvhuDC5lM
vrCBKdmaXhwMaTGppgFNl9adu8yBRQ6I2BN21hQejyfmCvHTsoN6u1oeYccL+K+D
lTK2dMCmXLixcQIJjS9sGA8/TXjmiY65RzIdXl0UHeV4JUtRREMUB1HeXJWCxMEt
4KQXb5tk21vZ+BTf9kRirnCtFT+Wudyo73M2XGKbR3mzHc2jY+RjEb71J0hUmh8s
h4kVhuMMkEFjNUwU017kEYQpmXMMRwCyUydc7BoV6hZXnUx4ROTpc6pDv2IZkeLB
bGiE
=adiR
-END PGP SIGNATURE-


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 9 April 2014 12:27:13 GMT-04:00, Tamas Blummer ta...@bitsofproof.com wrote:
A border router that is not able to serve blocks is still protecting
consensus rules, that SPVs do not.
If the network would only consist of SPV nodes only then e.g. a
majority coalition of miner could increase their reward at will.

Archives need a different solution.

Any collective group that has a majority of hashing power will have no major 
issues running enough nodes that follow their rules to make SPV insecure anyway.

There's no good reason not to have SPV security nodes distribute block chain 
data, particularly block headers. It helps provide redundancy in the network 
topology and helps provide more resources for full nodes to sync up faster. For 
instance in a network with a large number of partial UTXO set nodes if those 
nodes are forwarding block data to each other they can get enough data to 
become fully fledged full nodes without putting all the load on the existing 
full nodes.  This is a good thing.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCgA6BQJTRYdYMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhdwRB/46MAw7OHwnVkLHOD0g
Y4X6p1/QHgRisJIgpG2Y4nGVeAjOFleQWe4PWS4Wwdr4u0BDGPmJompiR3A99CaL
MUPnxJhtdiUsomn6kI704f5pkrqslQGLzejWFb7/9WuQtvGm8RwnzIs7uAqKasni
KJMn3jmqFIUcCEy9dePUdmMySCQj8qSxjGDdwZnwf8BZSdSLqd8dYiN0jQi/aA1I
2hWWyyDK9V9yQ8peAg+1dfg754Tc76lj3mEQOD39Wu3Klb9mBF3+wQCW2tJYEj2E
stzeOdFZsUNUIOKFb6mo0IoUipPOvrAkfm91ais+FIwlCf+k5KcwmAUXpV45rLHm
egCr
=2vMf
-END PGP SIGNATURE-


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Tamas Blummer
Block header has to be available in SPV and also in an UTXO only storing core 
node, so why not serve it if bandwith allows.

Serving any additional information like known peer adresses or known full 
blocks is certainly beneficial and should be offered if at hand.

Regards,

Tamas Blummer
http://bitsofproof.com

On 09.04.2014, at 19:46, Peter Todd p...@petertodd.org wrote:

 Signed PGP part
 
 
 On 9 April 2014 12:27:13 GMT-04:00, Tamas Blummer ta...@bitsofproof.com 
 wrote:
 A border router that is not able to serve blocks is still protecting
 consensus rules, that SPVs do not.
 If the network would only consist of SPV nodes only then e.g. a
 majority coalition of miner could increase their reward at will.
 
 Archives need a different solution.
 
 Any collective group that has a majority of hashing power will have no major 
 issues running enough nodes that follow their rules to make SPV insecure 
 anyway.
 
 There's no good reason not to have SPV security nodes distribute block chain 
 data, particularly block headers. It helps provide redundancy in the network 
 topology and helps provide more resources for full nodes to sync up faster. 
 For instance in a network with a large number of partial UTXO set nodes if 
 those nodes are forwarding block data to each other they can get enough data 
 to become fully fledged full nodes without putting all the load on the 
 existing full nodes.  This is a good thing.
 
 
 



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mike Hearn
The right way to start with this, if anyone cares, is to add
instrumentation to existing SPV wallet apps to report back to home base how
long they are running for, how much disk space / RAM they have, and
possibly what kind of hardware.

I *strongly* suspect that the vast majority of SPV wallets are not left
running permanently, and run on laptops where battery life is at a premium.
These people will never want to run full nodes.

Sorry. I don't think it will ever make sense to run full nodes on consumer
hardware again. Our time is much better spent on optimising so it's cheaper
for full node operators to run them on cheap virtualised servers.
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 9 April 2014 13:50:03 GMT-04:00, Tamas Blummer ta...@bitsofproof.com wrote:
Block header has to be available in SPV and also in an UTXO only
storing core node, so why not serve it if bandwith allows.

Serving any additional information like known peer adresses or known
full blocks is certainly beneficial and should be offered if at hand.

Big security advantages too. For instance if an attacker hacks, say, 10℅ of 
hashing power the next step for them to attack SPV clients is to try to Sybil 
attack them so they won't find out about the longer chain. The fewer providers 
of block chain data there are out there the easier that attack is - just 
simultaneously DoS a bunch of nodes, perhaps by a low-bandwidth exploit like 
the bloom io or division by zero DoS attacks. This is much harder to pull off 
if every SPV client is passing around block headers.

Similarly by passing around full blocks the attacker has a harder time knocking 
other miners off the network. Regardless of whether or not a miner's peers are 
fully validating chain data they still have the data they need to mine the next 
block and thus extend the longest correct chain.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCgA6BQJTRYu+MxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwheuQCADUyClLOLP1xpG1000l
uzcfPTZuIXTpzOAmYHKs/MSb6mph/Shsu0/94eW7npQNSVeZC8wQQZ1oFQ9j1GJc
SKViYJfn5yMdNvMkaWazhC0r3jxxF0AI7oy2KlnSjasfczfOQuYICJadTCwvUHrb
GrKVDbgsKNzZYYKn86vF4hsLwtJN4moeqX85TYN1DC7//7hgNywA73Xt2/gdwfqe
LOsD4nS7mUQObQd6TcLwXDDNEGTrdS572jdYH5sykwZjPH+wqwcm2WKTnIULsJR0
OwGUi505AKJJnLcEmZ/kGbCmKB+xJ5kExjlExtcUPrJlc+xqubhnnGCMjBiGCXSY
kYCK
=HXRZ
-END PGP SIGNATURE-


--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 8:00 PM, Mike Hearn m...@plan99.net wrote:

 The right way to start with this, if anyone cares, is to add
 instrumentation to existing SPV wallet apps to report back to home base how
 long they are running for, how much disk space / RAM they have, and
 possibly what kind of hardware.

 I *strongly* suspect that the vast majority of SPV wallets are not left
 running permanently, and run on laptops where battery life is at a premium.
 These people will never want to run full nodes.


Bitcoins stands or falls with people running full nodes.

If no one wants to volunteer resources to support the network anymore,
we'll have failed.

Sorry. I don't think it will ever make sense to run full nodes on consumer
 hardware again. Our time is much better spent on optimising so it's cheaper
 for full node operators to run them on cheap virtualised servers.


Most consumer hardware is much more powerful than 'cheap virtualized
servers'. More memory, disks are cheap, and at least in the Netherlands
home bandwidth is much cheaper than server bandwidth.

Also: any optimization that helps running on cheap servers will also help
running it on consumer hardware. It's not the one or the other.

Wladimir
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Kevin

On 4/9/2014 11:29 AM, Wladimir wrote:

Hello,

This is primarily aimed at developers of SPV wallets.

The recently reported decrease in number of full nodes could have 
several reasons, one of them that less people are running Bitcoin Core 
for the wallet because the other wallets are getting ahead in both 
features and useability.


It's great to see innovation in wallets, but it's worrying that the 
number of full nodes decreases.


It may be that lots of people would support the network by running a 
full node, but don't want to go through the trouble of installing 
bitcoin core separately (and get confused because it's a wallet, too).


Hence I'd like to explore the idea of adding an option to popular SPV 
wallets, to spin a bitcoind process in the background. This could be 
pretty much transparent to the user - it would sync in the background, 
the wallet could show statistics about the node, but is not dependent 
on it.


In exchange the user would get increased (full node level) security, 
as the SPV wallet would have a local trusted node.


Does this sound like a good idea?

Is there any way that Bitcoin Core can help to accomedate this 
'embedded' usage? Specific Interfaces, special builds - maybe add a 
walletless bitcoind build to gitian - bindings, dlls, etc?


Wladimir



--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees


___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
I personally like the ida.  Are you talking about a flag that could 
toggle this in the background mode or recoding for in the background use?



--
Kevin

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Justus Ranvier
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/09/2014 06:19 PM, Wladimir wrote:
 If no one wants to volunteer resources to support the network
 anymore, we'll have failed.

If the security of the network depends on a broken incentive model,
then fix the design of the network so that economics works for you
instead of against you.



- -- 
Support online privacy by using email encryption whenever possible.
Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTRZLiAAoJECoisBQbQ4v0RRMIAKL84ju4b9XAls9sOxQGrLMr
xifQDhThCQKC4/qkOmGU8zseIdKRgXEq4auMF9/0BlgoSxsBcKynlRH2fFtYY7ol
sdjCy/5dk+MBu3K1GCvNn/cuGkpIJJxrom/9riSiaPtivE5ncl7cyW5hFqf2MzRd
dF6q937ocgBd8d2VuOQleQ9N5gue1+du77soxfp8oY7dXNdwk5ZrngeYxz1umsjQ
lBAI/3JYkKVhU5Rte3deJcMHe6xA+neIzvcfWbOZYrkdwWE+jLSKnDkDkCOXJnuP
vOI8CAteaFctV2mfgXX2CmNDWVeFsyCwoQwbnCmuE/uiM5y235PcTrsyr6U+Zzs=
=oQPK
-END PGP SIGNATURE-


0x1B438BF4.asc
Description: application/pgp-keys
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 8:35 PM, Justus Ranvier justusranv...@gmail.comwrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 On 04/09/2014 06:19 PM, Wladimir wrote:
  If no one wants to volunteer resources to support the network
  anymore, we'll have failed.

 If the security of the network depends on a broken incentive model,
 then fix the design of the network so that economics works for you
 instead of against you.


My solution would be quick to implement and could help increase the number
of nodes in the short term. It will also smooth the way towards splitting
off the wallet from Bitcoin Core, by giving people a choice what wallet
they use with their full node.

That doesn't preclude looking for longer-term solutions that change the
incentive structure, but that is much more difficult and risky. I don't
believe it's a matter of 'fixing that instead' in a few hours, though I
nevertheless look forward to your pull request.

Wladimir
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 11:35 AM, Justus Ranvier justusranv...@gmail.com wrote:
 If the security of the network depends on a broken incentive model,
 then fix the design of the network so that economics works for you
 instead of against you.

Who says anything about a broken incentive model. You've made past
claims about resource requirements that I think made no sense and then
failed to defend them when they were challenge.

With suitable software improvements running a full node could be done
in as little as a few gigabytes in disk space (e.g. cost 25-50 cents),
and as 50-100kbit/sec bandwidth in and out ongoing, and a moderate
amount of ram. Power costs are already just a few cents per month.  By
far the greatest cost is the figuring out and setting up part, which
bundling could fix. The exact resources could be tunable to what the
users are willing and able to contribute.

If improved marginal security and privacy in addition to supporting
the network is not enough incentive to overcome costs like these then
Bitcoin is already doomed.  I think that fundamental costs aren't an
issue at all, just implementation warts and education are.

Part of asking the question is feeling out which improvements need to
happen first, and what the prospects of getting the bundling going
once those improvements are made.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Justus Ranvier
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/09/2014 06:50 PM, Gregory Maxwell wrote:
 Who says anything about a broken incentive model. You've made past 
 claims about resource requirements that I think made no sense and
 then failed to defend them when they were challenge.

Anyone reading the archives of the list will see about triple the
number of people independently confirming the resource usage problem
than they will see denying it, so I'm not particularly worried.


- -- 
Support online privacy by using email encryption whenever possible.
Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTRZhsAAoJECoisBQbQ4v0iD8H/A1RZyLJN6f+zt5x78CFgRdp
zqZua3NBwUN2L9mI/PU1NADtxoXgkq49XusHRc+bjLu17VxGMUOsmB+AeK3VU4sN
BVC1qIcWCa+OPMkgnMFrdydz4OGxX5xiJoCNsqveZIEYc8nQoCDfsn/fwqL50/Ct
NfmPzh/cW7uUZ+h2bBgduGD3fzpQBlnnswAbHHX3FbBh9MvcaCfROeXXqUCedWj9
H2qVCNWTkIFJksJkqyF3f+KfedOTIyUwflx/0niBvzXcP9w4oK+WpBApHACWxiSA
H3KHYNF0s2/fs+mIkabPLRX1PIUk15ln29FapoDvlHW9jzDE92x6JFnMppI4rEs=
=ILks
-END PGP SIGNATURE-


0x1B438BF4.asc
Description: application/pgp-keys
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 6:09 PM, Thomas Voegtlin thoma...@gmx.de wrote:

 Le 09/04/2014 17:54, Gregory Maxwell a écrit :

  Sadly today Electrum requires more than a full node, it requires a
  number of large additional indexes over what a full node has and
  pruning is precluded. I don't think that increasing the resource
  utilization of the node is a good way to go there for the purposes
  expressed here. (not that electrum couldn't be used here, but not
  unmodified without the resource usage increasing route)
 

 Electrum uses two large indexes:

  address - utxo

 (patricia tree, aka ultimate blockchain compression, see thread
 started by Alan Reiner in the bitcointalk forum)


Thanks for the explanation.

Adding a RPC call for a address - utxo query wouldn't be a big deal. It
has been requested before for other purposes as well, all the better if it
helps for interaction with Electrum.

Spent history would be involve a much larger index, and it's not likely
that will end up in bitcoin

Wladimir
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier justusranv...@gmail.com wrote:
 Anyone reading the archives of the list will see about triple the
 number of people independently confirming the resource usage problem
 than they will see denying it, so I'm not particularly worried.

The list has open membership, there is no particular qualification or
background required to post here. Optimal use of an information source
requires critical reading and understanding the limitations of the
medium. Counting comments is usually not a great way to assess
technical considerations on an open public forum.  Doubly so because
those comments were not actually talking about the same thing I am
talking about.

Existing implementations are inefficient in many known ways (and, no
doubt, some unknown ones). This list is about developing protocol and
implementations including improving their efficiency.  When talking
about incentives the costs you need to consider are the costs of the
best realistic option.  As far as I know there is no doubt from anyone
technically experienced that under the current network rules full
nodes can be operated with vastly less resources than current
implementations use, it's just a question of the relatively modest
implementation improvements.

When you argue that Bitcoin doesn't have the right incentives (and
thus something??) I retort that the actual resource _requirements_ are
for the protocol very low. I gave specific example numbers to enable
correction or clarification if I've said something wrong or
controversial. Pointing out that existing implementations are not that
currently as efficient as the underlying requirements and that some
large number of users do not like the efficiency of existing
implementations doesn't tell me anything I disagree with or didn't
already know. Whats being discussed around here contributes to
prioritizing improvements over the existing implementations.

I hope this clarifies something.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread slush
I believe there're plenty bitcoind instances running, but they don't have
configured port forwarding properly.There's uPNP support in bitcoind, but
it works only on simple setups.

Maybe there're some not yet considered way how to expose these *existing*
instances to Internet, to strenghten the network. Maybe just self-test
indicating the node is not reachable from outside (together with short
howto like in some torrent clients).

These days IPv6 is slowly deploying to server environments, but maybe
there's some simple way how to bundle ipv6 tunnelling into bitcoind so any
instance will become ipv6-reachable automatically?

Maybe there're other ideas how to improve current situation without needs
of reworking the architecture.

Marek


On Wed, Apr 9, 2014 at 9:33 PM, Gregory Maxwell gmaxw...@gmail.com wrote:

 On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier justusranv...@gmail.com
 wrote:
  Anyone reading the archives of the list will see about triple the
  number of people independently confirming the resource usage problem
  than they will see denying it, so I'm not particularly worried.

 The list has open membership, there is no particular qualification or
 background required to post here. Optimal use of an information source
 requires critical reading and understanding the limitations of the
 medium. Counting comments is usually not a great way to assess
 technical considerations on an open public forum.  Doubly so because
 those comments were not actually talking about the same thing I am
 talking about.

 Existing implementations are inefficient in many known ways (and, no
 doubt, some unknown ones). This list is about developing protocol and
 implementations including improving their efficiency.  When talking
 about incentives the costs you need to consider are the costs of the
 best realistic option.  As far as I know there is no doubt from anyone
 technically experienced that under the current network rules full
 nodes can be operated with vastly less resources than current
 implementations use, it's just a question of the relatively modest
 implementation improvements.

 When you argue that Bitcoin doesn't have the right incentives (and
 thus something??) I retort that the actual resource _requirements_ are
 for the protocol very low. I gave specific example numbers to enable
 correction or clarification if I've said something wrong or
 controversial. Pointing out that existing implementations are not that
 currently as efficient as the underlying requirements and that some
 large number of users do not like the efficiency of existing
 implementations doesn't tell me anything I disagree with or didn't
 already know. Whats being discussed around here contributes to
 prioritizing improvements over the existing implementations.

 I hope this clarifies something.


 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread slush
Another idea: Integrate torrent download of bootstrap.dat into bitcoind.
Normal user (especially a beginner) won't learn how to download bootstrap
separately and import it into bitcoind; he simply give up the
synchronization once he realize it takes too much time. From my experience
downloading the bootstrap significantly improves catching the blockchain,
which may attract some more users to run bitcoind.

Not sure about C++, but simple torrent client in python is like 30 lines of
code (using libtorrent).

Marek


On Wed, Apr 9, 2014 at 10:12 PM, slush sl...@centrum.cz wrote:

 I believe there're plenty bitcoind instances running, but they don't have
 configured port forwarding properly.There's uPNP support in bitcoind, but
 it works only on simple setups.

 Maybe there're some not yet considered way how to expose these *existing*
 instances to Internet, to strenghten the network. Maybe just self-test
 indicating the node is not reachable from outside (together with short
 howto like in some torrent clients).

 These days IPv6 is slowly deploying to server environments, but maybe
 there's some simple way how to bundle ipv6 tunnelling into bitcoind so any
 instance will become ipv6-reachable automatically?

 Maybe there're other ideas how to improve current situation without needs
 of reworking the architecture.

 Marek


 On Wed, Apr 9, 2014 at 9:33 PM, Gregory Maxwell gmaxw...@gmail.comwrote:

 On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier justusranv...@gmail.com
 wrote:
  Anyone reading the archives of the list will see about triple the
  number of people independently confirming the resource usage problem
  than they will see denying it, so I'm not particularly worried.

 The list has open membership, there is no particular qualification or
 background required to post here. Optimal use of an information source
 requires critical reading and understanding the limitations of the
 medium. Counting comments is usually not a great way to assess
 technical considerations on an open public forum.  Doubly so because
 those comments were not actually talking about the same thing I am
 talking about.

 Existing implementations are inefficient in many known ways (and, no
 doubt, some unknown ones). This list is about developing protocol and
 implementations including improving their efficiency.  When talking
 about incentives the costs you need to consider are the costs of the
 best realistic option.  As far as I know there is no doubt from anyone
 technically experienced that under the current network rules full
 nodes can be operated with vastly less resources than current
 implementations use, it's just a question of the relatively modest
 implementation improvements.

 When you argue that Bitcoin doesn't have the right incentives (and
 thus something??) I retort that the actual resource _requirements_ are
 for the protocol very low. I gave specific example numbers to enable
 correction or clarification if I've said something wrong or
 controversial. Pointing out that existing implementations are not that
 currently as efficient as the underlying requirements and that some
 large number of users do not like the efficiency of existing
 implementations doesn't tell me anything I disagree with or didn't
 already know. Whats being discussed around here contributes to
 prioritizing improvements over the existing implementations.

 I hope this clarifies something.


 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development



--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 10:12 PM, slush sl...@centrum.cz wrote:

 Maybe there're other ideas how to improve current situation without needs
 of reworking the architecture.


Nothing I've proposed here would require larger changes to the architecture
then were already planned. After SPV lands we are going to split off the
wallet, and that will need an interface to an bitcoind to allow 'running
with full node'. If that can be generalized to be useful for other (SPV)
clients as well, that would be useful, hence I asked for input.

It of course doesn't preclude also looking for other solutions.

Wladimir
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Wladimir
On Wed, Apr 9, 2014 at 10:31 PM, slush sl...@centrum.cz wrote:

 Another idea: Integrate torrent download of bootstrap.dat into bitcoind.
 Normal user (especially a beginner) won't learn how to download bootstrap
 separately and import it into bitcoind; he simply give up the
 synchronization once he realize it takes too much time. From my experience
 downloading the bootstrap significantly improves catching the blockchain,
 which may attract some more users to run bitcoind.

 Not sure about C++, but simple torrent client in python is like 30 lines
 of code (using libtorrent).


Parallel block download would be better for that. No need to involve
bittorrent.

But please let's not derail this thread, this is not about other solutions
for faster block download or such, let's keep it focused.

Wladimir
--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Laszlo Hanyecz

On Apr 9, 2014, at 8:12 PM, slush sl...@centrum.cz wrote:

 
 These days IPv6 is slowly deploying to server environments, but maybe there's 
 some simple way how to bundle ipv6 tunnelling into bitcoind so any instance 
 will become ipv6-reachable automatically?
 

Teredo is available by default on Microsoft systems and it's actually very 
common to see Teredo addresses as peers in torrents - this should work for 
bitcoin too, though I'm not sure if an app needs to set special flags to gain 
access to it.. there are probably some security settings around it.  In the US, 
ATT/CenturyLink provide IPv6 by way of 6RD for DSL customers, Comcast has 
native IPv6 on residential (but not business) cable modems, and of course those 
who want to can always set up with a tunnel broker like Hurricane Electric - 
they even let you use your own IPv6 addresses.  IPv6 is great, but having an 
application running its own tunnels would not be a good way to leverage it.

Probably what's keeping a lot of them from being reachable is that most people 
just plug their CPE into a NAT router (without IPv6).  Teredo can help here 
though.
Putting an IPv6 checkbox in the settings with a link to an explanation might 
help with education and result in smarter operators who can make their nodes 
reachable.  A built in checker would be even better - something like a 
checklist showing red/green for different aspects of reachability 
(v4/teredo/v6).

-Laszlo



--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Mark Friedenbach
I've advocated for this in the past, and reasonable counter-arguments I
was presented with are: (1) bittorrent is horribly insecure - it would
be easy to DoS the initial block download if that were the goal, and (2)
there's a reasonable pathway to doing this all in-protocol, so there's
no reason to introduce external dependencies.

On 04/09/2014 01:31 PM, slush wrote:
 Another idea: Integrate torrent download of bootstrap.dat into bitcoind.
 Normal user (especially a beginner) won't learn how to download
 bootstrap separately and import it into bitcoind; he simply give up the
 synchronization once he realize it takes too much time. From my
 experience downloading the bootstrap significantly improves catching the
 blockchain, which may attract some more users to run bitcoind.
 
 Not sure about C++, but simple torrent client in python is like 30 lines
 of code (using libtorrent).
 
 Marek
 
 
 On Wed, Apr 9, 2014 at 10:12 PM, slush sl...@centrum.cz
 mailto:sl...@centrum.cz wrote:
 
 I believe there're plenty bitcoind instances running, but they don't
 have configured port forwarding properly.There's uPNP support in
 bitcoind, but it works only on simple setups.
 
 Maybe there're some not yet considered way how to expose these
 *existing* instances to Internet, to strenghten the network. Maybe
 just self-test indicating the node is not reachable from outside
 (together with short howto like in some torrent clients).
 
 These days IPv6 is slowly deploying to server environments, but
 maybe there's some simple way how to bundle ipv6 tunnelling into
 bitcoind so any instance will become ipv6-reachable automatically?
 
 Maybe there're other ideas how to improve current situation without
 needs of reworking the architecture.
 
 Marek
 
 
 On Wed, Apr 9, 2014 at 9:33 PM, Gregory Maxwell gmaxw...@gmail.com
 mailto:gmaxw...@gmail.com wrote:
 
 On Wed, Apr 9, 2014 at 11:58 AM, Justus Ranvier
 justusranv...@gmail.com mailto:justusranv...@gmail.com wrote:
  Anyone reading the archives of the list will see about triple the
  number of people independently confirming the resource usage
 problem
  than they will see denying it, so I'm not particularly worried.
 
 The list has open membership, there is no particular
 qualification or
 background required to post here. Optimal use of an information
 source
 requires critical reading and understanding the limitations of the
 medium. Counting comments is usually not a great way to assess
 technical considerations on an open public forum.  Doubly so because
 those comments were not actually talking about the same thing I am
 talking about.
 
 Existing implementations are inefficient in many known ways (and, no
 doubt, some unknown ones). This list is about developing
 protocol and
 implementations including improving their efficiency.  When talking
 about incentives the costs you need to consider are the costs of the
 best realistic option.  As far as I know there is no doubt from
 anyone
 technically experienced that under the current network rules full
 nodes can be operated with vastly less resources than current
 implementations use, it's just a question of the relatively modest
 implementation improvements.
 
 When you argue that Bitcoin doesn't have the right incentives (and
 thus something??) I retort that the actual resource
 _requirements_ are
 for the protocol very low. I gave specific example numbers to enable
 correction or clarification if I've said something wrong or
 controversial. Pointing out that existing implementations are
 not that
 currently as efficient as the underlying requirements and that some
 large number of users do not like the efficiency of existing
 implementations doesn't tell me anything I disagree with or didn't
 already know. Whats being discussed around here contributes to
 prioritizing improvements over the existing implementations.
 
 I hope this clarifies something.
 
 
 --
 Put Bad Developers to Shame
 Dominate Development with Jenkins Continuous Integration
 Continuously Automate Build, Test  Deployment
 Start a new project now. Try Jenkins in the cloud.
 http://p.sf.net/sfu/13600_Cloudbees
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 mailto:Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development
 
 
 
 
 
 

Re: [Bitcoin-development] Bitcoind-in-background mode for SPV wallets

2014-04-09 Thread Gregory Maxwell
On Wed, Apr 9, 2014 at 1:36 PM, Mark Friedenbach m...@monetize.io wrote:
 (2) there's a reasonable pathway to doing this all in-protocol, so there's
 no reason to introduce external dependencies.

Not just a speculative pathway. Pieter implemented headers first:
https://github.com/sipa/bitcoin/tree/headersfirst  and it was
everything we hoped it would be— it easily can saturate residential
broadband, produces less load hot-spotting, copes with unreliable
peers, etc. It's now just slowly making its way into Bitcoin core.

--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development