Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:

[Mark Friedenbach]

It's not the pool operator's business what software the miner is running
to select transactions for his block, so long as the miner follows the
template and doesn't generate invalid blocks. We already discussed
invalid blocks, and checking the template requires parsing the coinbase
transaction and verifying the merkleroot. The most expensive operations
are the hashes in the merkleroot verification, but you have to do that
in stratum too because of the extranonce, right?

On 06/19/2014 01:55 PM, slush wrote:
> Miner issues are just one thing what came to my mind. What about
> validating transactions? How the pool can be sure that miner is running
> up to date bitcoind which do full validation of transactions? Not
> talking that if every miner takes his own set of transaction, pool need
> to calculate merkle root for every submit, to check its correctness.

--
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:

[slush]

Miner issues are just one thing what came to my mind. What about validating
transactions? How the pool can be sure that miner is running up to date
bitcoind which do full validation of transactions? Not talking that if
every miner takes his own set of transaction, pool need to calculate merkle
root for every submit, to check its correctness.

I don't think it *cannot* be done, it is just extremely hard and there's no
economic motivation for such complication on pool side. Just see current
pools, they are full of security and stability bugs even while using such
trivial protocol like Stratum...

slush

On Thu, Jun 19, 2014 at 10:39 PM, Mark Friedenbach  wrote:

> Do you need to do full validation? There's an economic cost to mining
> invalid blocks, and even if that were acceptable there's really no
> reason to perform such an attack. The result would be similar to a block
> withholding attack, but unlike block withholding it would be trivially
> detectable if/when full validation was performed.
>
> To protect yourself and to detect incorrect mining software you could
> stochastically validate a randomly selected sample of shares, as your
> hardware requirements allow.
>
> On 06/19/2014 01:26 PM, slush wrote:
> > With GBT, simply hashing the block header is not enough, because pool
> > cannot rely on anything provided by the client. Its not only about
> > things like withdrawal attacks, but more about hidden bugs in various
> > miners. It is extremely hard to do full validation for *every* of
> > submitted shares.
>
--
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:

[slush]

On Thu, Jun 19, 2014 at 7:35 PM, Mike Hearn  wrote:
>
> My (fresh!) understanding is that the reason we don't see people using
> getblocktemplate to decentralise pools is because libblkmaker and other
> implementations don't actually support connecting your own node to the
> miners and choosing your own blocks, even though the protocol does.
>
>
Well, I don't want to start flamewar (and this topic has potential!), but
the *real* reason why there isn't such infrastructure is that although GBT
as a protocol *does* support of decentralized building of blocks, it is
*extremely* resource consuming to validate these shares on pool side.

With GBT, simply hashing the block header is not enough, because pool
cannot rely on anything provided by the client. Its not only about things
like withdrawal attacks, but more about hidden bugs in various miners. It
is extremely hard to do full validation for *every* of submitted shares.

Although I see benefits of GBT and I see limits of Stratum, I don't think
that supporting full GBT validation have economic sense for any medium
sized pool, because such pool would need multiply his running costs on
servers many times.

> It's part of a general trend wherein people look at all the things
that can be accomplished in an economy that has a division of labor*,
and see some misbehavior at the edges, and decide that rather than
fixing the misbehavior we should throw out the entire concept of labor
specialization.

Well written! This reminds me - what about Stratum + SPV validation on
miner side?

With SPV, miner cannot choose his own transactions, so it is not fully
decentralized, but at least miner can detect (in real time) two major pool
misbehaviours - selfish mining (building private blockchain) and chain
forking (not working on longest known blockchain).

I read such proposal about Stratum + SPV on reddit and I actually like it;
It removes major drawbacks of "centralized" Stratum mining, yet is gives
tools to miners to instantly switch to fallback pool when something wrong
is happening.

slush
--
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:

[Mark Friedenbach]

Do you need to do full validation? There's an economic cost to mining
invalid blocks, and even if that were acceptable there's really no
reason to perform such an attack. The result would be similar to a block
withholding attack, but unlike block withholding it would be trivially
detectable if/when full validation was performed.

To protect yourself and to detect incorrect mining software you could
stochastically validate a randomly selected sample of shares, as your
hardware requirements allow.

On 06/19/2014 01:26 PM, slush wrote:
> With GBT, simply hashing the block header is not enough, because pool
> cannot rely on anything provided by the client. Its not only about
> things like withdrawal attacks, but more about hidden bugs in various
> miners. It is extremely hard to do full validation for *every* of
> submitted shares.

--
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:

[Justus Ranvier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 06/19/2014 05:11 PM, Kevin wrote:
> Why do you want to punish pools?

It's part of a general trend wherein people look at all the things
that can be accomplished in an economy that has a division of labor*,
and see some misbehavior at the edges, and decide that rather than
fixing the misbehavior we should throw out the entire concept of labor
specialization.

Hating on labor specialization as a concept, rather than simply
finding solutions for specialist misbehavior, was the basis for scrypt
mining, PoS, and MaidSafe.


(*) http://www.econlib.org/library/Essays/rdPncl1.html

- -- 
Support online privacy by using email encryption whenever possible.
Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJTox/JAAoJEMP3uyY4RQ216F0IAKo26MEK/IrIlHMw4UYsWBWK
LWWLe4mfUb+I74ZHPzC2ZE7u6Lf6vAeeG/mLe8K/ls1qBJk9ae7bsA+DVvRn8LfW
Ir/seYtCCnLpxhHGbVtYOeWaS+WyOWMKBz1moOTJcgWwPPiL5BLk9SvaLqgy2GDV
fJeniqtkZ96Vzif1DNdQtiLfn9aJRL2Er0EO7VL4ojmaM3Bv6Z6l+e0eLVVh8DKe
u1Sp4UOpqJmVHJq+9EeMhdfmOqNWGUA5wFRiDsWfzUDHLkAlISM+sFFSD0CzO0RK
P5whGxo58bSMigbQYOfoTZqgKQefQeXAqtlnrLOLq9/EOZ/34cJET5Z0S/K/F5E=
=3KMH
-END PGP SIGNATURE-


0x38450DB5.asc
Description: application/pgp-keys
--
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:

[Mike Hearn]

>
> The issue is centralized transaction selection policies, which is
> entirely orthogonal. And the solution already exists: getblocktemplate.


My (fresh!) understanding is that the reason we don't see people using
getblocktemplate to decentralise pools is because libblkmaker and other
implementations don't actually support connecting your own node to the
miners and choosing your own blocks, even though the protocol does.

I've written up a blog post that I hope will go out on the Foundation blog
soon with some low hanging fruity ideas for miner decentralisation.

Sergio, I'd love to give you intelligent feedback but unfortunately reading
it made my brain explode :) Sorry!
--
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:

[Mark Friedenbach]

Sergio, why is preventing mining pools a good thing? The issue is not
mining pools, which provide a needed service in greatly reducing
variance beyond what any proposal like this can do.

The issue is centralized transaction selection policies, which is
entirely orthogonal. And the solution already exists: getblocktemplate.
We just need more or better infrastructure which makes use of this
technology to perform local transaction selection.

If you have a proposal for eliminating hosted mining while keeping
variance-reducing pools, that would be an interesting read.

On 06/19/2014 09:58 AM, Sergio Lerner wrote:
> I propose a setting that prevent mining pools AND reduce payoff variance
> which requires two changes: increasing the block-rate and changing the
> Bitcoin PoW (but still allowing current Bitcoin ASICs to work (as far as
> I know)). The block rate must be increased at least 20 times and block
> must still be near full (e.g. there must be at least 20 more
> transactions/second than there is today)
> 
> BlockPow is kind of PoW that only practically prevents mining pools for
> certain cryptocurrency settings based on concepts similar to parmacoin,
> but in a much simple degree. The idea is that if miners try to join a
> pool, then they incur in overhead of transmitting information and earn
> less than working solo. Let b (the payload) contain a full block. b must
> contain all the transactions and the header, and not only the
> transaction hashes. b should not hide anything. Let h be the block
> header (which contains the previous block hash and the Merkle tree root
> of the transactions). Let d be the difficulty. hash-block-length(b)
> returns the number of blocks processed by the hash function when fed
> with the block b. The target is divided by hash-block-length(b) so that
> the difficulty does not depend on the length of the block. Some other
> function can be used to encourage nodes to add more or less transactions.
> 
> Def: Basic BlockPoW concept
> 
> For each r in the nonce-range: if H ( H( r || b ) || h || r) ) < 2^-d/
> hash-block-length(b) then return r
> 
> return null
> 
> The header (h) is appended to the hashed message to allow SPV clients to
> verify the PoW without requiring the full block to be downloaded. Peers
> can send only (x,r,h) to SPV nodes, where x = H( r || b ), so they can
> verify the PoW. The verification procedure is obvious, and is skipped
> here. r is inserted at the beginning of the message to prevent pool
> administrators from keeping a secret mid-state of the hash function
> secret in order to prevent block stealing and also to force the miner to
> know b in the inner mining loop.
> 
> So now mining requires the knowledge of the block b to be mined, and b
> must be received at each block-chain epoch. This could create an
> incentive not to include any transaction and use an almost empty b,
> because that way the bandwidth requirements is decreased. But this
> incentive should not exists normally, since by including transactions
> the solo miner gets an additional revenue from fees, which is lost if
> the block is empty. Anyway, to prevent this possible incentive we can
> append to b a subset of previous blocks (e.g 100 blocks). The block
> subset to include could be derived from a peudo-random function seeded
> by the previous block hash. Then a node would still need to download
> part or all the block-chain in order to mine.
> 
> Now if the miner wants to be a dumb node and take part of a pool, then
> the current working unsolved block (the template) must be sent each time
> from the pool admin to each miner. If the pool admin hosts 1000 miners,
> then to serve them with fresh block templates he needs 1000 times more
> bandwidth that the miners, making this much more expensive. If miners
> create another network topology to distribute templates, they are
> incurring in the same overhead as being actively part of the
> cryptocurrency network, so they gain nothing.
> 
> For example, in a block-chain with a 5 seconds block-rate, such as in
> NimbleCoin , each block can be as large as 200
> Kbytes (100 tps are allowed). A miner will require the block template to
> be ready as fast as possible, say before 3 seconds, so as not to loose
> more than 60% of the times the transaction fees present in the block
> template. This means that a pool admin serving 1000 clients must have a
> upload bandwidth of at least 60 Mbytes/sec, and load balance servers,
> because all miners will demand the block template at the same time and
> will compete for it.
> 
> The same miner, working solo, will not loose the 60% of block fees. If
> block fees are 10% of block reward, then solo miners earn 6% more than
> pool miners. Also, having a block rate of 5 seconds allows solo miners
> to receive payments more often and so it reduces the payment variance.
> 
> This method to discourage mining pools only work as long as time is
> takes to transmit a block is comparable t

Re: [Bitcoin-development] BlockPow: A Practical Proposal to prevent mining pools AND reduce payoff variance:

[Kevin]

On 6/19/2014 12:58 PM, Sergio Lerner wrote:


I propose a setting that prevent mining pools AND reduce payoff 
variance which requires two changes: increasing the block-rate and 
changing the Bitcoin PoW (but still allowing current Bitcoin ASICs to 
work (as far as I know)). The block rate must be increased at least 20 
times and block must still be near full (e.g. there must be at least 
20 more transactions/second than there is today)


BlockPow is kind of PoW that only practically prevents mining pools 
for certain cryptocurrency settings based on concepts similar to 
parmacoin, but in a much simple degree. The idea is that if miners try 
to join a pool, then they incur in overhead of transmitting 
information and earn less than working solo. Let b (the payload) 
contain a full block. b must contain all the transactions and the 
header, and not only the transaction hashes. b should not hide 
anything. Let h be the block header (which contains the previous block 
hash and the Merkle tree root of the transactions). Let d be the 
difficulty. hash-block-length(b) returns the number of blocks 
processed by the hash function when fed with the block b. The target 
is divided by hash-block-length(b) so that the difficulty does not 
depend on the length of the block. Some other function can be used to 
encourage nodes to add more or less transactions.


Def: Basic BlockPoW concept

For each r in the nonce-range: if H ( H( r || b ) || h || r) ) < 2^-d/ 
hash-block-length(b) then return r


return null

The header (h) is appended to the hashed message to allow SPV clients 
to verify the PoW without requiring the full block to be downloaded. 
Peers can send only (x,r,h) to SPV nodes, where x = H( r || b ), so 
they can verify the PoW. The verification procedure is obvious, and is 
skipped here. r is inserted at the beginning of the message to prevent 
pool administrators from keeping a secret mid-state of the hash 
function secret in order to prevent block stealing and also to force 
the miner to know b in the inner mining loop.


So now mining requires the knowledge of the block b to be mined, and b 
must be received at each block-chain epoch. This could create an 
incentive not to include any transaction and use an almost empty b, 
because that way the bandwidth requirements is decreased. But this 
incentive should not exists normally, since by including transactions 
the solo miner gets an additional revenue from fees, which is lost if 
the block is empty. Anyway, to prevent this possible incentive we can 
append to b a subset of previous blocks (e.g 100 blocks). The block 
subset to include could be derived from a peudo-random function seeded 
by the previous block hash. Then a node would still need to download 
part or all the block-chain in order to mine.


Now if the miner wants to be a dumb node and take part of a pool, then 
the current working unsolved block (the template) must be sent each 
time from the pool admin to each miner. If the pool admin hosts 1000 
miners, then to serve them with fresh block templates he needs 1000 
times more bandwidth that the miners, making this much more expensive. 
If miners create another network topology to distribute templates, 
they are incurring in the same overhead as being actively part of the 
cryptocurrency network, so they gain nothing.


For example, in a block-chain with a 5 seconds block-rate, such as in 
NimbleCoin , each block can be as large as 200 
Kbytes (100 tps are allowed). A miner will require the block template 
to be ready as fast as possible, say before 3 seconds, so as not to 
loose more than 60% of the times the transaction fees present in the 
block template. This means that a pool admin serving 1000 clients must 
have a upload bandwidth of at least 60 Mbytes/sec, and load balance 
servers, because all miners will demand the block template at the same 
time and will compete for it.


The same miner, working solo, will not loose the 60% of block fees. If 
block fees are 10% of block reward, then solo miners earn 6% more than 
pool miners. Also, having a block rate of 5 seconds allows solo miners 
to receive payments more often and so it reduces the payment variance.


This method to discourage mining pools only work as long as time is 
takes to transmit a block is comparable to the block interval time, 
e.g. 20%. This seems not to be a problem since if the cryptocurrency 
becomes popular, then we can expect blocks to be near full, while if 
is is not, then nobody will care about mining pools.


For this method to work for Bitcoin, Bitcoin should reduce the block 
rate to at least 1 minute, and keep blocks of at least 10 Mbytes. Or 
go the NimbleCoin way, and reduce the block interval to 5 seconds. The 
sole reduction of the block rate from 10 minutes to 5 seconds would 
reduce notably the mining reward variance, which is the main reason 
miners don't mine solo.


BitcoinBlockPow

The basic BlockPoW is incompatible with Bi