Re: [Bitcoin-development] Protocol extensions
I may be missing something, but perhaps the simplistic method is the best. Start all nodes off with a certain level of trust. Lets choose an arbitrary number 5. If a node's trust level is high enough (lets say 10) forward transactions it sends you without checking them. If a node's trust level is low enough (lets say 0) discard any transactions they send you (i.e. don't forward them on). For nodes with a trust level between 1 and 9, forward without checking between 1 and 9 out of every 10 transactions. Check the others, if they are valid, increase the trust level by 1, if they are invalid decrease the trust level by 3. All of the numbers mentioned here (1-10, 1, 10, 1, 5, and 3) are arbitrary numbers that should be determined by the client or user-preferences instead of the protocol. This would allow for clients to have varying amounts of initial trust/paranoia about their peers. By decreasing the amount of trust faster than we increase it, we make it harder for untrustworthy clients to cheat us. By having a cut off point, we make it so that untrustworthy clients can not DDoS us. By randomly verifying some transactions in the beginning, we make it harder for a new client from DDoSing us, and we prevent our own trust level from being hurt too much for forwarding on invalid transactions. The only problem I can personally see with this system is that if Node A trusts Node B with a 10 and Node C connects to Node A, then Node C can send transactions that are invalid to Node C via Node A without Node C being any the wiser. This would be stopped fairly quickly as Node B would catch on and stop forwarding transactions, but it would be a problem for new Nodes. This could be fixed (somewhat) by having a message that says not to trust a particular node. --Zell Faze --- On Thu, 12/22/11, Andy Parkins andypark...@gmail.com wrote: From: Andy Parkins andypark...@gmail.com Subject: Re: [Bitcoin-development] Protocol extensions To: Joel Joonatan Kaartinen joel.kaarti...@gmail.com Cc: bitcoin-development@lists.sourceforge.net Date: Thursday, December 22, 2011, 9:46 AM On 2011 December 22 Thursday, Joel Joonatan Kaartinen wrote: On Thu, 2011-12-22 at 11:52 +, Andy Parkins wrote: Why should they have to? Joining the network as a node is very low cost to the other nodes. You can't force any node not to be lazy, since their option is to disconnect themselves. As to maliciousness, that is defended against because when a node negative announces a transaction, that transaction is going to be checked (note that there is still no implicit trust) -- if a node is incorrectly negative-announcing then it can justifiably be kicked. a node that is not doing any checking themselves can not reliably forward failed verifications without getting the blame for doing faulty work. Those nodes would then have the incentive not to relay the failed verifications. This ends up making it important to know which nodes will be checking transactions or not so you don't isolate yourself from other nodes that are also checking transactions. Yes; I appreciate that. It's the very point I'm making. A node can choose what work to do, and should have a way of forwarding the results of that work to other nodes. Transaction verifification is the main one. Once a negative-announce message exists, it wouldn't be hard to have the other two you need as well: positive-announce and neutral-announce. At present we have only neutral-announce. However, as the need for super nodes and distributed verification gets bigger, having the forwarder able to offer an opinion on the quality of a transaction seems ideal to me. Dishonesty will get you isolated pretty quickly if you use positive-announce and negative- announce to lie. The problem with this is that it requires a web of trust as well as a web of connections. The only way to gain an advantage from this classified forwarding is if you have some way of assigning enough trust so that you can forward a classified transaction _without_ checking it yourself. That doesn't sound like an easy problem though. Andy -- Dr Andy Parkins andypark...@gmail.com -Inline Attachment Follows- -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev -Inline Attachment Follows- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Protocol extensions
Agree, but even before that, we will meet problems of the current 1MB/10min limit. The calculations from the scalability link surely indicates that there are 2 options for scalability either go for trusted supernodes backed by huge hardware resources or something else would be needed. The supernode approach is simple and easy to implement, but it also breaks a lot of the nice features about bitcoin. So if we want bitcoin to stay p2p we need to deploy other strategies. The hash space partitioning is one of them. And the nice thing is that it can be made to scale even for a javascript based validating and fully connected client running on a smartphone in a bitcoin future with billions of clients and transactions, and still it does not exclude you from running a trusted supernode either. Cheers, M On 21/12/2011, at 18:17, Jordan Mack wrote: I think it would be a lot more than that. According to the Scalability page (https://en.bitcoin.it/wiki/Scalability) if Bitcoin took over all credit card transactions, that would be about 1.14GB per block. I believe that is 58.5PB per year. (6*24*365*1.14/1024) This would also mean the distribution of 2MB of block data per second, which doesn't include broadcast overhead. On 12/21/2011 12:50 AM, Michael Grønager wrote: when bitcoin takes over all credit card transactions (!), and even before that, we will meet a scalability problem. The blockchain will grow rapidly, (1MB/10min or 50GB/yr) -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Protocol extensions
On 2011 December 21 Wednesday, Christian Decker wrote: Supernodes will be those nodes that verify all transactions and make them available to miners. Since miners will become more and more specialized these supernodes are likely to be owned by the miners themself. To be a miner either you need to verify all the transactions you include (otherwise others might be able to find an error in your block and thus drop it) or have someone that verifies them for you. In the end I think we'll end up with a hierarchical network, with the miners/supernodes tighly interconnected at the top and the lightweight clients that simply verify transactions (or their inputs to be precise) that are destined for them at the bottom. A thought occurred to me. We already run a decentralised system, but it's done by making everyone duplicate all other work. There is no fundamental reason why all work needs to be duplicated though. What about this: every node randomly chooses whether to verify any particular transaction. If we assume the network is large and the random factor is correctly chosen, then we can still guarantee that every transaction is verified. Then, we simply add a protocol message that is a negative-announce transaction. That is to say, we give nodes a way of telling other nodes that they think a transaction is invalid. The other nodes are then free to verify _that_ assertion and forward the negative-announce. Miners can then listen for negative-announcements and use them to decide were to dedicate their verification efforts. They then don't need to verify all (or perhaps even any) transactions themselves and can dedicate their processing power to mining. (I've actually mentioned this idea before, but that time I was using it as a double-spend prevention method). Andy -- Dr Andy Parkins andypark...@gmail.com signature.asc Description: This is a digitally signed message part. -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Protocol extensions
On Sat, Dec 17, 2011, at 05:27 PM, Jordan Mack wrote: I don't like the idea of a header only client, unless this is just an interim action until the full block chain is downloaded in the background. Development of these types of clients is probably inevitable, but I believe that this breaks the most fundamental aspects of Bitcoin's security model. If a client has only headers, it cannot do full verification, and it is trusting the data from random anonymous peers. A headers-only client is much better than trusting anyone, since an attacker needs 50% of the network's computational power to trick such clients. For everyone to keep being a full node, hardware costs would need to constantly go down enough for all nodes to be able to handle enough transactions to meet demand. If hardware doesn't become cheap enough quickly enough, either some people would be unable to handle being full nodes, or the max block size wouldn't rise enough to meet demand and transaction fees would become noncompetitive. -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Protocol extensions
The whole point of having headers built at a constant size and generation rate is to minimize the amount of data needed to understand of the blockchain while simultaneously maximizing integrity/security in the presence of untrusted nodes. Barring the 50%-attack, you only need a couple honest nodes out of 50 to stay safe (as long as you're waiting for your 6 confirmations). In fact, I would argue that a full node (Satoshi client), has the same level of security as a headers-only client... because they both base *all* their verification decisions on computations that end with comparing hashes to the longest-chain headers. In the case that an attacker figures out how to isolate your node entirely and start feeing you poisoned blocks, then you are vulnerable with any kind of node, full or lightweight. I don't see where the reduced security is. The only issue I see is that a truly light-weight, headers-only node will be having to download an entire block to get one transaction it needs. This would be significantly alleviated if nodes can start requesting merkle-trees directly, even without merkle-branch-pruning. If a node can ask for a tx and the tx-hash-list of the block that incorporated that tx, he can easily verify his tx against his no-need-to-trust-anyone headers, and doesn't have to download MBs for every one. As for blockchain pruning... I think it's absolutely critical to find a way to do this, /for all nodes/. I am swayed by Dan Kaminsky's scalability warnings, and my instinct tells me that leaving full verification to a select few deep-pockets nodes in the future opens up all sorts of centralization/power-corporation issues that is contrary to the Bitcoin concept. It is in everyone's best interest to make it as easy as possible for /anyone/ to act as a full node (if possible). As such, I believe that the current system of minimizing TxOut size is the right one. TxIns take up 0 bytes space in the long-run when taking into account any blockchain pruning/snapshot idea (except for nLocktime/seq transactions where the TxIn might have to be saved). -Alan On 12/18/2011 12:09 PM, theymos wrote: On Sat, Dec 17, 2011, at 05:27 PM, Jordan Mack wrote: I don't like the idea of a header only client, unless this is just an interim action until the full block chain is downloaded in the background. Development of these types of clients is probably inevitable, but I believe that this breaks the most fundamental aspects of Bitcoin's security model. If a client has only headers, it cannot do full verification, and it is trusting the data from random anonymous peers. A headers-only client is much better than trusting anyone, since an attacker needs50% of the network's computational power to trick such clients. For everyone to keep being a full node, hardware costs would need to constantly go down enough for all nodes to be able to handle enough transactions to meet demand. If hardware doesn't become cheap enough quickly enough, either some people would be unable to handle being full nodes, or the max block size wouldn't rise enough to meet demand and transaction fees would become noncompetitive. -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Protocol extensions
Has anyone considered 'snapshot' frames (blocks). Message to node: getsnapshot: hash Node responds with a 'block' message. Then the hash for that particular snapshot is hardcoded into the sourcecode. It would replace the checkpoints and use the last hash in that list. Validating blocks is pretty fast right up until block 135k, which is where time taken balloons and starts become exponentially slower. As blockchain grows linearly, resources needed grows exponentially if you think about it. From: Alan Reiner etothe...@gmail.com To: bitcoin-development@lists.sourceforge.net Sent: Sunday, December 18, 2011 6:06 PM Subject: Re: [Bitcoin-development] Protocol extensions The whole point of having headers built at a constant size and generation rate is to minimize the amount of data needed to understand of the blockchain while simultaneously maximizing integrity/security in the presence of untrusted nodes. Barring the 50%-attack, you only need a couple honest nodes out of 50 to stay safe (as long as you're waiting for your 6 confirmations). In fact, I would argue that a full node (Satoshi client), has the same level of security as a headers-only client... because they both base all their verification decisions on computations that end with comparing hashes to the longest-chain headers. In the case that an attacker figures out how to isolate your node entirely and start feeing you poisoned blocks, then you are vulnerable with any kind of node, full or lightweight. I don't see where the reduced security is. The only issue I see is that a truly light-weight, headers-only node will be having to download an entire block to get one transaction it needs. This would be significantly alleviated if nodes can start requesting merkle-trees directly, even without merkle-branch-pruning. If a node can ask for a tx and the tx-hash-list of the block that incorporated that tx, he can easily verify his tx against his no-need-to-trust-anyone headers, and doesn't have to download MBs for every one. As for blockchain pruning... I think it's absolutely critical to find a way to do this, for all nodes. I am swayed by Dan Kaminsky's scalability warnings, and my instinct tells me that leaving full verification to a select few deep-pockets nodes in the future opens up all sorts of centralization/power-corporation issues that is contrary to the Bitcoin concept. It is in everyone's best interest to make it as easy as possible for anyone to act as a full node (if possible). As such, I believe that the current system of minimizing TxOut size is the right one. TxIns take up 0 bytes space in the long-run when taking into account any blockchain pruning/snapshot idea (except for nLocktime/seq transactions where the TxIn might have to be saved). -Alan On 12/18/2011 12:09 PM, theymos wrote: On Sat, Dec 17, 2011, at 05:27 PM, Jordan Mack wrote: I don't like the idea of a header only client, unless this is just an interim action until the full block chain is downloaded in the background. Development of these types of clients is probably inevitable, but I believe that this breaks the most fundamental aspects of Bitcoin's security model. If a client has only headers, it cannot do full verification, and it is trusting the data from random anonymous peers. A headers-only client is much better than trusting anyone, since an attacker needs 50% of the network's computational power to trick such clients. For everyone to keep being a full node, hardware costs would need to constantly go down enough for all nodes to be able to handle enough transactions to meet demand. If hardware doesn't become cheap enough quickly enough, either some people would be unable to handle being full nodes, or the max block size wouldn't rise enough to meet demand and transaction fees would become noncompetitive. -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Bitcoin
Re: [Bitcoin-development] Protocol extensions
2011/12/17, theymos they...@mm.st: My preferred solution for handling scalability in the future is to have lightweight clients download only headers and Merkle trees (which are both small and easy to distribute), and then require senders to contact recipients directly in order to transmit their transactions. Then lightweight clients never need full blocks to build their balances, and full nodes don't have to handle expensive queries from lightweight clients. This idea is really interesting. Is there any drawback I don't see? -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Protocol extensions
While using DHT for storage of the block chain is an intriguing concept, I do not see how it is feasible. As Gavin noted, DHT is a system that is difficult to impossible to guarantee against data loss or manipulation. Even if we found a way to store the block chain in DHT, how would transactions be verified? As Gavin noted, you could ask the network, but cannot necessarily trust the peers you are connected to. Verification of the full block chain allows the client to trust no one. I also do not see how DHT would solve the problem of scalability in regards to broadcast messages, although I am definitely interested in the concept. -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Protocol extensions
On Sat, Dec 17, 2011 at 7:44 PM, Jordan Mack jordanm...@parhelic.com wrote: While using DHT for storage of the block chain is an intriguing concept, I do not see how it is feasible. As Gavin noted, DHT is a system that is difficult to impossible to guarantee against data loss or manipulation. Even if we found a way to store the block chain in DHT, how would transactions be verified? As Gavin noted, you could ask the network, but cannot necessarily trust the peers you are connected to. Verification of the full block chain allows the client to trust no one. Well, the block chain data itself is internally self-validating. As long as you know the latest block's hash -- a big if -- there is no problem downloading all other block chain data from DHT or any other untrusted source. In a malicious case, you would notice latest-hash differs from non-malicious and wind up downloading multiple chains, when walking hashes backwards through a DHT/lookup table. So, a bit more work but nothing fundamentally less secure _on a trust basis_. Of course, I was focusing on data validation, which ignores other factors such as DoS'ing the DHT. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
