[Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:
I propose a setting that prevent mining pools AND reduce payoff variance which requires two changes: increasing the block-rate and changing the Bitcoin PoW (but still allowing current Bitcoin ASICs to work (as far as I know)). The block rate must be increased at least 20 times and block must still be near full (e.g. there must be at least 20 more transactions/second than there is today) BlockPow is kind of PoW that only practically prevents mining pools for certain cryptocurrency settings based on concepts similar to parmacoin, but in a much simple degree. The idea is that if miners try to join a pool, then they incur in overhead of transmitting information and earn less than working solo. Let b (the payload) contain a full block. b must contain all the transactions and the header, and not only the transaction hashes. b should not hide anything. Let h be the block header (which contains the previous block hash and the Merkle tree root of the transactions). Let d be the difficulty. hash-block-length(b) returns the number of blocks processed by the hash function when fed with the block b. The target is divided by hash-block-length(b) so that the difficulty does not depend on the length of the block. Some other function can be used to encourage nodes to add more or less transactions. Def: Basic BlockPoW concept For each r in the nonce-range: if H ( H( r || b ) || h || r) ) 2^-d/ hash-block-length(b) then return r return null The header (h) is appended to the hashed message to allow SPV clients to verify the PoW without requiring the full block to be downloaded. Peers can send only (x,r,h) to SPV nodes, where x = H( r || b ), so they can verify the PoW. The verification procedure is obvious, and is skipped here. r is inserted at the beginning of the message to prevent pool administrators from keeping a secret mid-state of the hash function secret in order to prevent block stealing and also to force the miner to know b in the inner mining loop. So now mining requires the knowledge of the block b to be mined, and b must be received at each block-chain epoch. This could create an incentive not to include any transaction and use an almost empty b, because that way the bandwidth requirements is decreased. But this incentive should not exists normally, since by including transactions the solo miner gets an additional revenue from fees, which is lost if the block is empty. Anyway, to prevent this possible incentive we can append to b a subset of previous blocks (e.g 100 blocks). The block subset to include could be derived from a peudo-random function seeded by the previous block hash. Then a node would still need to download part or all the block-chain in order to mine. Now if the miner wants to be a dumb node and take part of a pool, then the current working unsolved block (the template) must be sent each time from the pool admin to each miner. If the pool admin hosts 1000 miners, then to serve them with fresh block templates he needs 1000 times more bandwidth that the miners, making this much more expensive. If miners create another network topology to distribute templates, they are incurring in the same overhead as being actively part of the cryptocurrency network, so they gain nothing. For example, in a block-chain with a 5 seconds block-rate, such as in NimbleCoin http://nimbleCoin.org, each block can be as large as 200 Kbytes (100 tps are allowed). A miner will require the block template to be ready as fast as possible, say before 3 seconds, so as not to loose more than 60% of the times the transaction fees present in the block template. This means that a pool admin serving 1000 clients must have a upload bandwidth of at least 60 Mbytes/sec, and load balance servers, because all miners will demand the block template at the same time and will compete for it. The same miner, working solo, will not loose the 60% of block fees. If block fees are 10% of block reward, then solo miners earn 6% more than pool miners. Also, having a block rate of 5 seconds allows solo miners to receive payments more often and so it reduces the payment variance. This method to discourage mining pools only work as long as time is takes to transmit a block is comparable to the block interval time, e.g. 20%. This seems not to be a problem since if the cryptocurrency becomes popular, then we can expect blocks to be near full, while if is is not, then nobody will care about mining pools. For this method to work for Bitcoin, Bitcoin should reduce the block rate to at least 1 minute, and keep blocks of at least 10 Mbytes. Or go the NimbleCoin way, and reduce the block interval to 5 seconds. The sole reduction of the block rate from 10 minutes to 5 seconds would reduce notably the mining reward variance, which is the main reason miners don't mine solo. BitcoinBlockPow The basic BlockPoW is incompatible with Bitcoin ASICs as is but it can be made partially compatible with some tweaks: The value b is replaced by a a a subset or an
Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:
On 6/19/2014 12:58 PM, Sergio Lerner wrote: I propose a setting that prevent mining pools AND reduce payoff variance which requires two changes: increasing the block-rate and changing the Bitcoin PoW (but still allowing current Bitcoin ASICs to work (as far as I know)). The block rate must be increased at least 20 times and block must still be near full (e.g. there must be at least 20 more transactions/second than there is today) BlockPow is kind of PoW that only practically prevents mining pools for certain cryptocurrency settings based on concepts similar to parmacoin, but in a much simple degree. The idea is that if miners try to join a pool, then they incur in overhead of transmitting information and earn less than working solo. Let b (the payload) contain a full block. b must contain all the transactions and the header, and not only the transaction hashes. b should not hide anything. Let h be the block header (which contains the previous block hash and the Merkle tree root of the transactions). Let d be the difficulty. hash-block-length(b) returns the number of blocks processed by the hash function when fed with the block b. The target is divided by hash-block-length(b) so that the difficulty does not depend on the length of the block. Some other function can be used to encourage nodes to add more or less transactions. Def: Basic BlockPoW concept For each r in the nonce-range: if H ( H( r || b ) || h || r) ) 2^-d/ hash-block-length(b) then return r return null The header (h) is appended to the hashed message to allow SPV clients to verify the PoW without requiring the full block to be downloaded. Peers can send only (x,r,h) to SPV nodes, where x = H( r || b ), so they can verify the PoW. The verification procedure is obvious, and is skipped here. r is inserted at the beginning of the message to prevent pool administrators from keeping a secret mid-state of the hash function secret in order to prevent block stealing and also to force the miner to know b in the inner mining loop. So now mining requires the knowledge of the block b to be mined, and b must be received at each block-chain epoch. This could create an incentive not to include any transaction and use an almost empty b, because that way the bandwidth requirements is decreased. But this incentive should not exists normally, since by including transactions the solo miner gets an additional revenue from fees, which is lost if the block is empty. Anyway, to prevent this possible incentive we can append to b a subset of previous blocks (e.g 100 blocks). The block subset to include could be derived from a peudo-random function seeded by the previous block hash. Then a node would still need to download part or all the block-chain in order to mine. Now if the miner wants to be a dumb node and take part of a pool, then the current working unsolved block (the template) must be sent each time from the pool admin to each miner. If the pool admin hosts 1000 miners, then to serve them with fresh block templates he needs 1000 times more bandwidth that the miners, making this much more expensive. If miners create another network topology to distribute templates, they are incurring in the same overhead as being actively part of the cryptocurrency network, so they gain nothing. For example, in a block-chain with a 5 seconds block-rate, such as in NimbleCoin http://nimbleCoin.org, each block can be as large as 200 Kbytes (100 tps are allowed). A miner will require the block template to be ready as fast as possible, say before 3 seconds, so as not to loose more than 60% of the times the transaction fees present in the block template. This means that a pool admin serving 1000 clients must have a upload bandwidth of at least 60 Mbytes/sec, and load balance servers, because all miners will demand the block template at the same time and will compete for it. The same miner, working solo, will not loose the 60% of block fees. If block fees are 10% of block reward, then solo miners earn 6% more than pool miners. Also, having a block rate of 5 seconds allows solo miners to receive payments more often and so it reduces the payment variance. This method to discourage mining pools only work as long as time is takes to transmit a block is comparable to the block interval time, e.g. 20%. This seems not to be a problem since if the cryptocurrency becomes popular, then we can expect blocks to be near full, while if is is not, then nobody will care about mining pools. For this method to work for Bitcoin, Bitcoin should reduce the block rate to at least 1 minute, and keep blocks of at least 10 Mbytes. Or go the NimbleCoin way, and reduce the block interval to 5 seconds. The sole reduction of the block rate from 10 minutes to 5 seconds would reduce notably the mining reward variance, which is the main reason miners don't mine solo. BitcoinBlockPow The basic BlockPoW is incompatible with
Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:
Sergio, why is preventing mining pools a good thing? The issue is not mining pools, which provide a needed service in greatly reducing variance beyond what any proposal like this can do. The issue is centralized transaction selection policies, which is entirely orthogonal. And the solution already exists: getblocktemplate. We just need more or better infrastructure which makes use of this technology to perform local transaction selection. If you have a proposal for eliminating hosted mining while keeping variance-reducing pools, that would be an interesting read. On 06/19/2014 09:58 AM, Sergio Lerner wrote: I propose a setting that prevent mining pools AND reduce payoff variance which requires two changes: increasing the block-rate and changing the Bitcoin PoW (but still allowing current Bitcoin ASICs to work (as far as I know)). The block rate must be increased at least 20 times and block must still be near full (e.g. there must be at least 20 more transactions/second than there is today) BlockPow is kind of PoW that only practically prevents mining pools for certain cryptocurrency settings based on concepts similar to parmacoin, but in a much simple degree. The idea is that if miners try to join a pool, then they incur in overhead of transmitting information and earn less than working solo. Let b (the payload) contain a full block. b must contain all the transactions and the header, and not only the transaction hashes. b should not hide anything. Let h be the block header (which contains the previous block hash and the Merkle tree root of the transactions). Let d be the difficulty. hash-block-length(b) returns the number of blocks processed by the hash function when fed with the block b. The target is divided by hash-block-length(b) so that the difficulty does not depend on the length of the block. Some other function can be used to encourage nodes to add more or less transactions. Def: Basic BlockPoW concept For each r in the nonce-range: if H ( H( r || b ) || h || r) ) 2^-d/ hash-block-length(b) then return r return null The header (h) is appended to the hashed message to allow SPV clients to verify the PoW without requiring the full block to be downloaded. Peers can send only (x,r,h) to SPV nodes, where x = H( r || b ), so they can verify the PoW. The verification procedure is obvious, and is skipped here. r is inserted at the beginning of the message to prevent pool administrators from keeping a secret mid-state of the hash function secret in order to prevent block stealing and also to force the miner to know b in the inner mining loop. So now mining requires the knowledge of the block b to be mined, and b must be received at each block-chain epoch. This could create an incentive not to include any transaction and use an almost empty b, because that way the bandwidth requirements is decreased. But this incentive should not exists normally, since by including transactions the solo miner gets an additional revenue from fees, which is lost if the block is empty. Anyway, to prevent this possible incentive we can append to b a subset of previous blocks (e.g 100 blocks). The block subset to include could be derived from a peudo-random function seeded by the previous block hash. Then a node would still need to download part or all the block-chain in order to mine. Now if the miner wants to be a dumb node and take part of a pool, then the current working unsolved block (the template) must be sent each time from the pool admin to each miner. If the pool admin hosts 1000 miners, then to serve them with fresh block templates he needs 1000 times more bandwidth that the miners, making this much more expensive. If miners create another network topology to distribute templates, they are incurring in the same overhead as being actively part of the cryptocurrency network, so they gain nothing. For example, in a block-chain with a 5 seconds block-rate, such as in NimbleCoin http://nimbleCoin.org, each block can be as large as 200 Kbytes (100 tps are allowed). A miner will require the block template to be ready as fast as possible, say before 3 seconds, so as not to loose more than 60% of the times the transaction fees present in the block template. This means that a pool admin serving 1000 clients must have a upload bandwidth of at least 60 Mbytes/sec, and load balance servers, because all miners will demand the block template at the same time and will compete for it. The same miner, working solo, will not loose the 60% of block fees. If block fees are 10% of block reward, then solo miners earn 6% more than pool miners. Also, having a block rate of 5 seconds allows solo miners to receive payments more often and so it reduces the payment variance. This method to discourage mining pools only work as long as time is takes to transmit a block is comparable to the block interval time, e.g. 20%. This seems not to be a problem since if
Re: [Bitcoin-development] instant confirmation via payment protocol backwards compatible proto buffer extension
Supporting it in the protocol is easy. Building such a thing: that's hard. Decentralised automated reputation systems are complex and subtle. Bitcoin is valuable as a protocol because it is truly decentralized. The complexity involved in building this system was expansive, but I think we can all agree it was worth the trouble. With this particular topic of instant transactions it seems we have to be very careful about pushing Bitcoin in a centralized direction for the sake of a simple quick solution. Building an automated system to solve the instant transaction problem will be difficult, but also well worth the effort, and exactly like you're saying Mike, I just want to make sure the door is left open protocol wise for a robust solution in the future. On Wed, Jun 18, 2014 at 9:09 AM, Mike Hearn m...@plan99.net wrote: I think that's true if you assume that the instant provider list is based on a by hand created list of accepted instant providers. That's how VISA works now and that's why I was asking for an approach where the trusted_instant_providers list is scalable because that seems very dangerous. Supporting it in the protocol is easy. Building such a thing: that's hard. Decentralised automated reputation systems are complex and subtle. I don't feel strongly about whether the field should be optional or repeated, 100% of implementations in the forseeable future would just look at the first item and ignore the rest. But if later someone did crack this problem it would lead to a simple upgrade path. So perhaps you're right and the protobuf should allow multiple signatures. It means a new sub-message to wrap the pki_type, pki_data and signature fields into one, and then making that repeated. Up to Lawrence. -- HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing Easy Data Exploration http://p.sf.net/sfu/hpccsystems___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:
The issue is centralized transaction selection policies, which is entirely orthogonal. And the solution already exists: getblocktemplate. My (fresh!) understanding is that the reason we don't see people using getblocktemplate to decentralise pools is because libblkmaker and other implementations don't actually support connecting your own node to the miners and choosing your own blocks, even though the protocol does. I've written up a blog post that I hope will go out on the Foundation blog soon with some low hanging fruity ideas for miner decentralisation. Sergio, I'd love to give you intelligent feedback but unfortunately reading it made my brain explode :) Sorry! -- HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing Easy Data Exploration http://p.sf.net/sfu/hpccsystems___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/19/2014 05:11 PM, Kevin wrote: Why do you want to punish pools? It's part of a general trend wherein people look at all the things that can be accomplished in an economy that has a division of labor*, and see some misbehavior at the edges, and decide that rather than fixing the misbehavior we should throw out the entire concept of labor specialization. Hating on labor specialization as a concept, rather than simply finding solutions for specialist misbehavior, was the basis for scrypt mining, PoS, and MaidSafe. (*) http://www.econlib.org/library/Essays/rdPncl1.html - -- Support online privacy by using email encryption whenever possible. Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJTox/JAAoJEMP3uyY4RQ216F0IAKo26MEK/IrIlHMw4UYsWBWK LWWLe4mfUb+I74ZHPzC2ZE7u6Lf6vAeeG/mLe8K/ls1qBJk9ae7bsA+DVvRn8LfW Ir/seYtCCnLpxhHGbVtYOeWaS+WyOWMKBz1moOTJcgWwPPiL5BLk9SvaLqgy2GDV fJeniqtkZ96Vzif1DNdQtiLfn9aJRL2Er0EO7VL4ojmaM3Bv6Z6l+e0eLVVh8DKe u1Sp4UOpqJmVHJq+9EeMhdfmOqNWGUA5wFRiDsWfzUDHLkAlISM+sFFSD0CzO0RK P5whGxo58bSMigbQYOfoTZqgKQefQeXAqtlnrLOLq9/EOZ/34cJET5Z0S/K/F5E= =3KMH -END PGP SIGNATURE- 0x38450DB5.asc Description: application/pgp-keys -- HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing Easy Data Exploration http://p.sf.net/sfu/hpccsystems___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:
Do you need to do full validation? There's an economic cost to mining invalid blocks, and even if that were acceptable there's really no reason to perform such an attack. The result would be similar to a block withholding attack, but unlike block withholding it would be trivially detectable if/when full validation was performed. To protect yourself and to detect incorrect mining software you could stochastically validate a randomly selected sample of shares, as your hardware requirements allow. On 06/19/2014 01:26 PM, slush wrote: With GBT, simply hashing the block header is not enough, because pool cannot rely on anything provided by the client. Its not only about things like withdrawal attacks, but more about hidden bugs in various miners. It is extremely hard to do full validation for *every* of submitted shares. -- HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing Easy Data Exploration http://p.sf.net/sfu/hpccsystems ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:
On Thu, Jun 19, 2014 at 7:35 PM, Mike Hearn m...@plan99.net wrote: My (fresh!) understanding is that the reason we don't see people using getblocktemplate to decentralise pools is because libblkmaker and other implementations don't actually support connecting your own node to the miners and choosing your own blocks, even though the protocol does. Well, I don't want to start flamewar (and this topic has potential!), but the *real* reason why there isn't such infrastructure is that although GBT as a protocol *does* support of decentralized building of blocks, it is *extremely* resource consuming to validate these shares on pool side. With GBT, simply hashing the block header is not enough, because pool cannot rely on anything provided by the client. Its not only about things like withdrawal attacks, but more about hidden bugs in various miners. It is extremely hard to do full validation for *every* of submitted shares. Although I see benefits of GBT and I see limits of Stratum, I don't think that supporting full GBT validation have economic sense for any medium sized pool, because such pool would need multiply his running costs on servers many times. It's part of a general trend wherein people look at all the things that can be accomplished in an economy that has a division of labor*, and see some misbehavior at the edges, and decide that rather than fixing the misbehavior we should throw out the entire concept of labor specialization. Well written! This reminds me - what about Stratum + SPV validation on miner side? With SPV, miner cannot choose his own transactions, so it is not fully decentralized, but at least miner can detect (in real time) two major pool misbehaviours - selfish mining (building private blockchain) and chain forking (not working on longest known blockchain). I read such proposal about Stratum + SPV on reddit and I actually like it; It removes major drawbacks of centralized Stratum mining, yet is gives tools to miners to instantly switch to fallback pool when something wrong is happening. slush -- HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing Easy Data Exploration http://p.sf.net/sfu/hpccsystems___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:
Miner issues are just one thing what came to my mind. What about validating transactions? How the pool can be sure that miner is running up to date bitcoind which do full validation of transactions? Not talking that if every miner takes his own set of transaction, pool need to calculate merkle root for every submit, to check its correctness. I don't think it *cannot* be done, it is just extremely hard and there's no economic motivation for such complication on pool side. Just see current pools, they are full of security and stability bugs even while using such trivial protocol like Stratum... slush On Thu, Jun 19, 2014 at 10:39 PM, Mark Friedenbach m...@monetize.io wrote: Do you need to do full validation? There's an economic cost to mining invalid blocks, and even if that were acceptable there's really no reason to perform such an attack. The result would be similar to a block withholding attack, but unlike block withholding it would be trivially detectable if/when full validation was performed. To protect yourself and to detect incorrect mining software you could stochastically validate a randomly selected sample of shares, as your hardware requirements allow. On 06/19/2014 01:26 PM, slush wrote: With GBT, simply hashing the block header is not enough, because pool cannot rely on anything provided by the client. Its not only about things like withdrawal attacks, but more about hidden bugs in various miners. It is extremely hard to do full validation for *every* of submitted shares. -- HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing Easy Data Exploration http://p.sf.net/sfu/hpccsystems___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
