Re: [Bitcoin-development] Punishing empty blocks?
Peter, I like the idea of being able to know what fees to expect from different
miners (it is like a service description / SLA for their service), but I would
prefer a more distributed discovery mechanism for the information on the fees
(Spent 10 years on Grid Computing...).
Miners could e.g. include a pointer to a webpage (or even their min fee) in the
coinbase (encoded properly, like the /P2SH/ string for BIP0016). That way
clients could look it up them selves or you could create sites accumulating
this information from the chain it self.
So something like :
const char* service_sla =
|https://my_ubercool_asic_mining_pool/sla.php|;
COINBASE_FLAGS std::vectorunsigned char(service_sla,
service_sla+strlen(service_sla));
The format of the sla.php page should then be specified too - but it could be a
json-rpc call returning a json object like (as result):
{
sla_version: 0.1,
accept_no_fee_tx: false,
min_fee: 5,
big_tx_fee: 1, // extra fee pr kb
}
I guess miners could work out a more suitable set of fees...
Seems like this calls for a BIP ?
/M
On 28/05/2012, at 16:54, Peter Vessenes wrote:
One of the issues here though is that it would be nice if miners published
their own tx rules -- it might be hard to impute them from data.
I had started a thread about this on bitcoin.org some time ago, and I don't
recall what the general outcome was.
I had imagined an open service whereby a miner could publish a short string
in their conbase tying to the service and the service would have different
metadata, including the miner's transaction guarantees.
We offered to host this before, and would still be willing to host such a
service.
Peter
On Sat, May 26, 2012 at 7:52 AM, Stefan Thomas m...@justmoon.de wrote:
Zooko is spot on - slower confirmations will give people a reason to set
higher fees. As soon as fees reach a level where they matter, even
botnet operators will be looking into ways of including transactions for
some extra profit.
In the meantime slightly slower confirmations aren't a problem. Consider
that even if it takes four blocks to get your transaction included
instead of one, once it is included, you still benefit from every new
block in terms of security. So if you're looking for six confirmations
for example, even a three block delay will only be a 50% delay for you.
And of course there are techniques for instant transactions which
continue to be refined and improved.
As for the proposed solutions: Punishing 1-tx blocks is complete and
utter nonsense. It's trivial to include a bogus second transaction.
Any additional challenges towards miners like hashes of the previous
block are at best useless. If I was running a botnet, I'd just grab that
hash from a website (pretty good chance Blockchain.info will have it :P)
or mining pool or wherever and keep going undeterred. At worst they may
affect scalability one day. You might imagine a peer-to-peer network of
miners who for cost reasons don't download all blocks anymore, but
verify only a percentage of them at random. They might then exchange
messages about invalid blocks including a proof (invalid tx, merkle
branch) why the block is invalid. This is just one idea, the point is
that assumptions about what a legitimate miner looks like may not always
hold in the future.
Finally, there is an ethical aspect as well. If a miner wishes not to
include my transaction that is his choice. He has no more an obligation
to sell his service to me than I have to buy it from him. If I really,
really want him to include my transaction I will have to offer to pay more.
If we as developers think that confirmations are too slow or that more
blocks should include transactions, then the right measures would be:
- Educating users about the relationship between confirmation speed and fees
- Raising the default transaction fee
Every market has a supply curve, so it is economically to be expected
that there will be some miners who don't include transactions, simply
because they are at that end of the supply curve where it is not worth
it for them to sell their service. All markets must have a certain
tension - there must be miners who don't include transactions for there
to be users who want their transactions included more quickly. In other
words there must be somebody not confirming if confirmations are to have
value. If you interfere with that all you'll accomplish is keep
transaction fees below market level, which will make the transition from
inflation-financed hashing to transaction-financed hashing more painful
and disruptive.
Cheers,
Stefan
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security,
Re: [Bitcoin-development] Punishing empty blocks?
On Tuesday, May 29, 2012 3:05:18 PM Peter Vessenes wrote: 1) Germane to the original conversation, anything hard to implement will not get implemented by miners. Without my got-tired-of-waiting-for-someone-to-merge-it coinbaser branch, anything modifying the coinbase is hard to implement. 2) Coinbase is hard-limited to 100 bytes; this has to include space for voting as well as extra nonce, etc. So, I'm not sure that a full URL is a good plan. Rather, I would suggest a 20 byte keyhash, which allows the owner to broadcast a full URI out-of-band. 1) They shall prepend \mi: to the url to designate it as a url for miner info, and append a trailing \ to the url How about a simple prefix to the fixed-size keyhash? Perhaps MFR= (Mining Fee Rules) 2) The url given in the coinbase shall have http:// prepended to it before processing. I would recommend miners use https, with a specified SSL keyhash in the URI (so we don't need to pay for a proper SSL cert). 3) The destination may be a redirect (to allow short URLs), or may deliver content Clients should simply be required to follow the relevant HTTP specification. 4) The content-type returned by the final site post-redirect shall be either (preferred text/json) or text/plain or text/html text/plain and text/html are just wrong and don't make any sense here. Inre: Luke's complaint about JSON, it is the language of the web. There is no easier format for both computers and humans to read, and in this case, it includes extensibility, which is nice, since we have no idea how miners will wish to divvy up their services; I think one would need to make a strong case against JSON for a specific reason to not choose it by default. Bitcoin isn't the web, it's a complicated script-based cryptocurrency. Everything in the Bitcoin protocol requires a computer's interpretation for humans, and there's no reason to stray from this default. Also, JSON is not extensible in any of the ways needed for this specific purpose. 4) The text of the document delivered shall be a JSON format dictionary, and shall include at minimum the following fields: 'min_fee', 'pool_name', and 'last_modified' Optional fields can be determined over time as necessary by the mining community Last Modified and other caching rules are dealt with in the relevant HTTP specification... 5) The Service Level Agreement isn't binding, but miners who implement it are expected to make a best efforts attempt to follow it. While it doesn't make sense to give it the full legal force of a contract, I think it should be expressed as a MUST in the BIP. Generally a miner would occasionally publish the \mi:\ when they had updated their SLA, or just every so often, but the canonical location would be the final destination URL from the redirects. The coinbase advertisement MUST be part of every coinbase mined by the miner, or there's no reliable way to prove which blocks are theirs. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
On Tuesday, May 29, 2012 3:28:56 PM Peter Vessenes wrote: I don't understand what the 20 byte keyhash is. Can you elucidate? 20 byte keyhashes are a fundamental building block of the Bitcoin protocol. ripemd160(sha256(ecdsaPubKey)) -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
OK, I have a few thoughts on this:
1) Germane to the original conversation, anything hard to implement will
not get implemented by miners.
2) Coinbase is hard-limited to 100 bytes; this has to include space for
voting as well as extra nonce, etc. So, I'm not sure that a full URL is a
good plan.
3) I'm a little fuzzy on the details of BIP governance; but I'm happy to
write one up and get my thoughts down, or someone who's more familiar could
do it, I suppose.
I propose the following spec:
periodically a miner may choose to publish a url through their coinbase as
follows:
1) They shall prepend \mi: to the url to designate it as a url for miner
info, and append a trailing \ to the url
2) The url given in the coinbase shall have http:// prepended to it before
processing.
3) The destination may be a redirect (to allow short URLs), or may deliver
content
4) The content-type returned by the final site post-redirect shall be
either (preferred text/json) or text/plain or text/html
4) The text of the document delivered shall be a JSON format dictionary,
and shall include at minimum the following fields: 'min_fee', 'pool_name',
and 'last_modified' Optional fields can be determined over time as
necessary by the mining community
5) The Service Level Agreement isn't binding, but miners who implement it
are expected to make a best efforts attempt to follow it.
So a valid coinbase could be:
/P2SH/\mi:goo.gl/mr2D\extra_nonce:2110
Generally a miner would occasionally publish the \mi:\ when they had
updated their SLA, or just every so often, but the canonical location would
be the final destination URL from the redirects.
Inre: Luke's complaint about JSON, it is the language of the web. There is
no easier format for both computers and humans to read, and in this case,
it includes extensibility, which is nice, since we have no idea how miners
will wish to divvy up their services; I think one would need to make a
strong case against JSON for a specific reason to not choose it by default.
Thoughts welcome!
Best,
Peter
On Tue, May 29, 2012 at 10:47 AM, Luke-Jr l...@dashjr.org wrote:
On Tuesday, May 29, 2012 8:52:49 AM Michael Grønager wrote:
The format of the sla.php page should then be specified too - but it
could
be a json-rpc call returning a json object like (as result): {
sla_version: 0.1,
accept_no_fee_tx: false,
min_fee: 5,
big_tx_fee: 1, // extra fee pr kb
}
I guess miners could work out a more suitable set of fees...
Please not JSON, and not hard-coded logic. Bitcoin already has a secure
scripting system - perhaps we can decide on an initial stack format and
run a
script retrieved from the URI?
--
Peter J. Vessenes
CEO, CoinLab
M: 206.595.9839
Skype: vessenes
Google+ https://plus.google.com/112885659993091300749
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
I suppose I mean that I don't understand how to reverse that into a URL when one is presented only with a block, or perhaps a coinbase in a transaction. Best, Peter On Tue, May 29, 2012 at 11:34 AM, Luke-Jr l...@dashjr.org wrote: On Tuesday, May 29, 2012 3:28:56 PM Peter Vessenes wrote: I don't understand what the 20 byte keyhash is. Can you elucidate? 20 byte keyhashes are a fundamental building block of the Bitcoin protocol. ripemd160(sha256(ecdsaPubKey)) -- Peter J. Vessenes CEO, CoinLab M: 206.595.9839 Skype: vessenes Google+ https://plus.google.com/112885659993091300749 -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
On Tuesday, May 29, 2012 3:36:34 PM Peter Vessenes wrote: I suppose I mean that I don't understand how to reverse that into a URL when one is presented only with a block, or perhaps a coinbase in a transaction. A new message can be added to the p2p relay network, similar to tx and alert broadcasts, that allow miners to publish/update their policy URI signed by the key in question. Counter-DDoS rules could decline to relay or store URIs for keys that haven't been published in - or achieved statistical significance in - the last N blocks. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
I see. That is undeniably more secure and bitcoin-y than my suggestion. It's also really a lot more work, especially in that it requires extra linkages between codebases that in my mind are largely separate. I'm just one voice, but I persist in believing that the 'lighter' solution, especially for something that may not be a particularly big problem in the bitcoin world is good -- it carries much less technical implementation debt going forward, and has a lower risk of sort of seizing up development with additional necessary code to worry about for those implementing to-spec clients. If that lighter solution turns out to be gameable, or has problems that require the full force of the bitcoin network and concepts, that would be the time to implement the improved version. That's just my approach, however. I worry that building in any additional requirements to the protocol or codebase adds significant cost to the network as a whole over the next 10 years. Peter On Tue, May 29, 2012 at 11:39 AM, Luke-Jr l...@dashjr.org wrote: On Tuesday, May 29, 2012 3:36:34 PM Peter Vessenes wrote: I suppose I mean that I don't understand how to reverse that into a URL when one is presented only with a block, or perhaps a coinbase in a transaction. A new message can be added to the p2p relay network, similar to tx and alert broadcasts, that allow miners to publish/update their policy URI signed by the key in question. Counter-DDoS rules could decline to relay or store URIs for keys that haven't been published in - or achieved statistical significance in - the last N blocks. -- Peter J. Vessenes CEO, CoinLab M: 206.595.9839 Skype: vessenes Google+ https://plus.google.com/112885659993091300749 -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
I disagree with a bunch of your points, but I'll wait on others to comment, except I will say that I don't understand what the 20 byte keyhash is. Can you elucidate? I am assuming major mining folks have written their own coinbasing facilities, but perhaps this is not the case -- if so, I agree that some work is necessary for such miners. Finally I will just comment that I am guided by the general perspective that many things about bitcoins are opt-in; therefore it makes sense to me put difficult work onto those who are motivated to do it, and keep things as easy as possible for the 'maybes' to participate -- hence small courtesies like allowing text/plain or text/html. Peter On Tue, May 29, 2012 at 11:18 AM, Luke-Jr l...@dashjr.org wrote: On Tuesday, May 29, 2012 3:05:18 PM Peter Vessenes wrote: 1) Germane to the original conversation, anything hard to implement will not get implemented by miners. Without my got-tired-of-waiting-for-someone-to-merge-it coinbaser branch, anything modifying the coinbase is hard to implement. 2) Coinbase is hard-limited to 100 bytes; this has to include space for voting as well as extra nonce, etc. So, I'm not sure that a full URL is a good plan. Rather, I would suggest a 20 byte keyhash, which allows the owner to broadcast a full URI out-of-band. 1) They shall prepend \mi: to the url to designate it as a url for miner info, and append a trailing \ to the url How about a simple prefix to the fixed-size keyhash? Perhaps MFR= (Mining Fee Rules) 2) The url given in the coinbase shall have http:// prepended to it before processing. I would recommend miners use https, with a specified SSL keyhash in the URI (so we don't need to pay for a proper SSL cert). 3) The destination may be a redirect (to allow short URLs), or may deliver content Clients should simply be required to follow the relevant HTTP specification. 4) The content-type returned by the final site post-redirect shall be either (preferred text/json) or text/plain or text/html text/plain and text/html are just wrong and don't make any sense here. Inre: Luke's complaint about JSON, it is the language of the web. There is no easier format for both computers and humans to read, and in this case, it includes extensibility, which is nice, since we have no idea how miners will wish to divvy up their services; I think one would need to make a strong case against JSON for a specific reason to not choose it by default. Bitcoin isn't the web, it's a complicated script-based cryptocurrency. Everything in the Bitcoin protocol requires a computer's interpretation for humans, and there's no reason to stray from this default. Also, JSON is not extensible in any of the ways needed for this specific purpose. 4) The text of the document delivered shall be a JSON format dictionary, and shall include at minimum the following fields: 'min_fee', 'pool_name', and 'last_modified' Optional fields can be determined over time as necessary by the mining community Last Modified and other caching rules are dealt with in the relevant HTTP specification... 5) The Service Level Agreement isn't binding, but miners who implement it are expected to make a best efforts attempt to follow it. While it doesn't make sense to give it the full legal force of a contract, I think it should be expressed as a MUST in the BIP. Generally a miner would occasionally publish the \mi:\ when they had updated their SLA, or just every so often, but the canonical location would be the final destination URL from the redirects. The coinbase advertisement MUST be part of every coinbase mined by the miner, or there's no reliable way to prove which blocks are theirs. -- Peter J. Vessenes CEO, CoinLab M: 206.595.9839 Skype: vessenes Google+ https://plus.google.com/112885659993091300749 -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
I like the concept except that it only works if every node connected to the miner enforces the rule (if it works). Once any one of the nodes forwards the block, other nodes see it coming from a node that can pass the challenge. I don't think any solution based on node queries will succeed, especially if it requires spontaneous super-majority-of-nodes acceptance. I think it's gotta be based on the block itself and each nodes' own info. If you could spontaneously get all miners to agree not to build off of anti-social blocks (however that is defined) , it would have a chance of making a difference, but individual miners would have an advantage building off the antisocial block because they only need to produce one to create the longest chain (and collect reward) while the miners following the rules need two blocks. --Sent from my overpriced smartphone On May 25, 2012 3:48 AM, Christian Decker decker.christ...@gmail.com wrote: How about a simple proof of work test? This one though does not ask for CPU work but asks the miner for a random old transaction. If the miner really stores the entire blockchain he will not have any problem answering to that getdata request, whereas a botnet would have to ask someone else for it, which could be detected if the response time deviates too much from what has been previously measured (compare it against getdata for the block they advertise). It's not perfect but it allows an estimate of whether it is a chainless miner. Regards, Chris -- Christian Decker On Fri, May 25, 2012 at 3:17 AM, Jeff Garzik jgar...@exmulti.com wrote: On Thu, May 24, 2012 at 8:57 PM, Luke-Jr l...@dashjr.org wrote: Block times are not accurate enough for that. The times in your log are very accurate, assuming your system clock is remotely accurate. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
We just implemented our own mining tool, soup-to-nuts, and I would say that the likely motivation for what I presume are botnet owners is just economic. It's a lot more work to make sure your merkleing and keeping up-to-date are happening than it is to just get an 80 byte header from a central server, and re-calc a single transaction merkle client-side. Not to mention the extra work to keep track of what version of getmemorypool output you're receiving work for in a broadly distributed pool. For what it's worth, we did this extra engineering work since we care about Bitcoin, but if I just wanted to pull value out of the ecosystem, we would have skipped it. The only solutions to this are economic solutions -- making such 'cheater' blocks less valuable, or increasing the value of the transactions. Also note that botnet operators likely care, in the end, about fiat currency, so going to 25 btc per block in what I think of as transaction fee subsidies won't necessarily impact this -- it's a matter of what happens to exchange rates vs generation rates that will matter. I think we also have to moderate this consideration against the rights (and arguable benefits) of someone wanting to build an express-delivery mining service, one that will provide provably faster certification for those adding a transaction fee of, say, 1 btc. My own experience now in the MMO world is that we have to carefully understand how we deal with griefers who control massive resources (compute or gold-farmers). It may not be a winning battle to choose a solution which harms the rest of the network in exchange for harming the griefers. This is definitely out of the box, but one solution might be to change the difficulty calculations to just ignore 1tx blocks; that would minimize impact on others to a great extent, and would let someone set up an express block service if they chose. I guess we'd have to settle on whether or not such blocks counted towards the issuance countdown as well. Or, we could allow only 1/10 generation fees on such blocks. Peter On Fri, May 25, 2012 at 9:44 AM, Alan Reiner etothe...@gmail.com wrote: I like the concept except that it only works if every node connected to the miner enforces the rule (if it works). Once any one of the nodes forwards the block, other nodes see it coming from a node that can pass the challenge. I don't think any solution based on node queries will succeed, especially if it requires spontaneous super-majority-of-nodes acceptance. I think it's gotta be based on the block itself and each nodes' own info. If you could spontaneously get all miners to agree not to build off of anti-social blocks (however that is defined) , it would have a chance of making a difference, but individual miners would have an advantage building off the antisocial block because they only need to produce one to create the longest chain (and collect reward) while the miners following the rules need two blocks. --Sent from my overpriced smartphone On May 25, 2012 3:48 AM, Christian Decker decker.christ...@gmail.com wrote: How about a simple proof of work test? This one though does not ask for CPU work but asks the miner for a random old transaction. If the miner really stores the entire blockchain he will not have any problem answering to that getdata request, whereas a botnet would have to ask someone else for it, which could be detected if the response time deviates too much from what has been previously measured (compare it against getdata for the block they advertise). It's not perfect but it allows an estimate of whether it is a chainless miner. Regards, Chris -- Christian Decker On Fri, May 25, 2012 at 3:17 AM, Jeff Garzik jgar...@exmulti.com wrote: On Thu, May 24, 2012 at 8:57 PM, Luke-Jr l...@dashjr.org wrote: Block times are not accurate enough for that. The times in your log are very accurate, assuming your system clock is remotely accurate. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats.
Re: [Bitcoin-development] Punishing empty blocks?
For what it is worth, I question whether this is a problem. Or, I guess I question whether the best solution to it isn't for people to start including more transaction fees. In fact, I'm not entirely sure that this problem doesn't actually *encourage* people to that solution, which would be very good if true. I would be more comfortable if the reward for mining were more commensurate with the value it provides. Ultimately, of course, that means that each transaction fee would have to be more of a proportion of the value *to the spender* of that transaction being included in the blockchain. (Aside: in order to convey to outsiders that miners are providing a useful service rather than gaining undeserved reward for wasting electricity, I refer to them as distributed transaction verification servers rather than miners whenever possible.) I'm pretty sure that — assuming there isn't some Bitcoin-killing disaster — transaction fees will eventually rise, but sooner might be better, especially with the first coinbase-halving looming. Perhaps people will be motivated to include transaction fees if they know that some miners don't bother to validate their transactions and others do. They may feel motivated to reward the miners that are serving them and punish the ones that are not. (Note: this wouldn't be a valid strategy on their part from a strictly game-theoretic perspective, but if they act on those motivations, then I don't care if it was rational or not.) Also, they may decide that they want to counteract the added delay which those no-transactions miners are adding to *all* transactions (with or without fee), by putting a fee on their transactions in order to make them take less long when they are processed by a miner which does process (some) transactions. Already this visualization, which I typically glance at a few times a day, usually shows a good separation with fee-included transactions sometimes doing much better than (some) free transactions: http://bitcoinstats.org/ However, this graph shows that the aggregate reward to the miners for processing transaction is minimal: http://blockchain.info/charts/transaction-fees?timespan=60daysshowDataPoints=falsedaysAverageString=1show_header=truescale=0address= You can see from the first visualization (assuming it is showing the typical pattern that I've seen) how you risk greater delay by sending your transaction without fees. The no-transactions miners push *all* transactions, fee or no-fee to the right. This may incentivize more people to change their transactions from red diamonds into blue circles, in order to move their transactions further to the left, even though the no-transactions miners are not currently discriminating among the two types. Therefore, the presence of those miners may help push the aggregate fees in that latter graph up, which is something I would very much like to see. Regards, Zooko -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Punishing empty blocks?
There appears to be some non-trivial mining power devoted to mining empty blocks. Even with satoshi's key observation -- hash a fixed 80-byte header, not the entire block -- some miners still find it easier to mine empty blocks, rather than watch the network for new transactions. Therefore I was wondering what people thought about a client implementation change: - Do not store or relay empty blocks, if time since last block X (where X = 60 minutes, perhaps) or even stronger, - Ensure latest block includes at least X percent of mempool unconfirmed TXs The former is easier to implement, though there is the danger that no-TX miners simply include a statically generated transaction or two. The latter might be considered problematic, as it might refuse to relay quickly found blocks. Comments? It wouldn't be a problem if these no-TX blocks were not already getting frequent (1 in 20). -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
I think you need the stronger change. Otherwise, the mystery miner could just put in a few transactions to himself to mask his block. His block would appear to be of some use while not being helpful. -Arthur On Thu, May 24, 2012 at 9:33 AM, Jeff Garzik jgar...@exmulti.com wrote: There appears to be some non-trivial mining power devoted to mining empty blocks. Even with satoshi's key observation -- hash a fixed 80-byte header, not the entire block -- some miners still find it easier to mine empty blocks, rather than watch the network for new transactions. Therefore I was wondering what people thought about a client implementation change: - Do not store or relay empty blocks, if time since last block X (where X = 60 minutes, perhaps) or even stronger, - Ensure latest block includes at least X percent of mempool unconfirmed TXs The former is easier to implement, though there is the danger that no-TX miners simply include a statically generated transaction or two. The latter might be considered problematic, as it might refuse to relay quickly found blocks. Comments? It wouldn't be a problem if these no-TX blocks were not already getting frequent (1 in 20). -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
I think the strong verification would go well if you add it along with an optimization that avoids rechecking transactions that have already been verified as valid. Any transactions it doesn't have to verify are from the pool, of course :) On Thu, May 24, 2012 at 7:33 PM, Jeff Garzik jgar...@exmulti.com wrote: There appears to be some non-trivial mining power devoted to mining empty blocks. Even with satoshi's key observation -- hash a fixed 80-byte header, not the entire block -- some miners still find it easier to mine empty blocks, rather than watch the network for new transactions. Therefore I was wondering what people thought about a client implementation change: - Do not store or relay empty blocks, if time since last block X (where X = 60 minutes, perhaps) or even stronger, - Ensure latest block includes at least X percent of mempool unconfirmed TXs The former is easier to implement, though there is the danger that no-TX miners simply include a statically generated transaction or two. The latter might be considered problematic, as it might refuse to relay quickly found blocks. Comments? It wouldn't be a problem if these no-TX blocks were not already getting frequent (1 in 20). -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
On Thu, May 24, 2012 at 1:13 PM, Joel Joonatan Kaartinen joel.kaarti...@gmail.com wrote: optimization that avoids rechecking transactions that have already been verified as valid. Any transactions it doesn't have to verify are from the pool, of course :) Work in this area is already progressing, though it is outside the scope of this proposal regarding lazy miners and empty blocks. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
On Thu, May 24, 2012 at 1:27 PM, Robert McKay rob...@mckay.com wrote: If miners wanted to continue mining empty blocks without bothering to monitor the Tx pool they would just switch to stuffing the empty blocks with a dummy transaction of their own to get round your new rules. Yes. This was stated in the original email. Once the block reward halves in a few months time then receiving transaction fees will probably become more important to the miner's profit and loss calculations and they'll spend the extra time to implement proper transaction processing. I suspect if we do nothing this particular issue will go away. Perhaps it could be helped along by publishing some example code to make it easier for them. At current rates it is potentially years before that point is reached -- years of degraded service for existing users. The ability to refuse transactions seems like an important part of the game theory of transaction pricing. Miners are supposed to be able to jack up transaction costs by declining to process no fee or too low fee (in their opinion) transactions.. the counter balance is that they are losing money by doing that and leaving more on the table for the next miner to score a block. I expect that in the future there will be other instances when people complain that the miners are being 'unfair' and that the rules should be changed in some way to lower transaction fees (ie: increase block size). If you see a rule change, you have misunderstood the proposal. This is an -implementation- change, which users and miners are free to accept or reject as part of their choice of software to use in the bitcoin ecosystem. As such, miners continue to be free to build upon empty blocks, and let those blocks become part of a useful chain. You would not simply /ban/ empty blocks completely, but avoid relaying top-of-chain empty blocks. Mining power and network collaborate to choose the best chain at that point -- perhaps even including those empty blocks. Clients will continue to follow the longest, strongest chain, even after this implementation change. An implementation change is a soft vote of choice by the user, not a hard requirement on all users. I think it should be legitimate not to publish a transaction to the p2p network at all.. in the future there will probably be lots of networks other than the p2p network.. right now we have the IPv6 network and the IPv4 network.. in the future there could be many other protocols and perhaps not all transactions will make it back to the old legacy ipv4 p2p network or into the mempool of bitcoin nodes on that network.. but they should still be able to get into the block chain. See above -- such behavior is perfectly fine. It should be noted that out of band (OOB) TXs, transited through third party means outside P2P network, would not cause _empty_ blocks, as the block chain will continue to have traffic for the foreseeable future. OOB TXs are a great idea, too. In a hyperscaled bitcoin future, OOB TXs might even be the norm. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
On Thursday, May 24, 2012 4:33:12 PM Jeff Garzik wrote: There appears to be some non-trivial mining power devoted to mining empty blocks. Even with satoshi's key observation -- hash a fixed 80-byte header, not the entire block -- some miners still find it easier to mine empty blocks, rather than watch the network for new transactions. Therefore I was wondering what people thought about a client implementation change: - Do not store or relay empty blocks, if time since last block X (where X = 60 minutes, perhaps) or even stronger, - Ensure latest block includes at least X percent of mempool unconfirmed TXs These are problematic for legitimate miners: 1) The freedom to reject transactions based on fees or spam filters, is severely restricted. As mentioned in other replies, this is an important point of Bitcoin's design. 1b) This punishes miners with superior transaction spam filtering. As with all spam filtering, it is often an arms race and therefore the filter rules must be kept private by the miners, and therefore cannot be disclosed for the validating clients to take into consideration. 2) For a few seconds after a new block is received, the new transaction merkle root(s) are not finished calculating. During this time, most miners are working on blank blocks with the new previousblockhash but no transactions. If those blocks are ignored, miners are forced to shutdown mining during this time. 3) As you mentioned, illegitimate miners can easily workaround these restrictions (even the second one, by flooding the network with their own transactions). This puts the legitimate miners at a disadvantage in their own search for valid blocks, unless they also come up with counter-measures themselves. The argument that these are not rule changes is flawed: 1) As of right now, 99% of the network runs a single client. Anything this client rejects does de facto become a rule change. 2) Even if there were a diverse ecosystem of clients in place, discouragement rules that potentially affect legitimate miners significantly mess with the odds of finding a block. 3) If legitimate miners do not adopt counter-rules to bypass these new restrictions, the illegitimate miners are left with an even larger percentage of blocks found. To summarize, I believe such a change as proposed would be very harmful to Bitcoin. Luke -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
On Thu, May 24, 2012 at 4:31 PM, Luke-Jr l...@dashjr.org wrote: These are problematic for legitimate miners: 1) The freedom to reject transactions based on fees or spam filters, is severely restricted. As mentioned in other replies, this is an important point of Bitcoin's design. 1b) This punishes miners with superior transaction spam filtering. As with all spam filtering, it is often an arms race and therefore the filter rules must be kept private by the miners, and therefore cannot be disclosed for the validating clients to take into consideration. This is simply not true given current available data, i.e. the current blockchain and ongoing not-spam transaction rate/pool. The argument that these are not rule changes is flawed: 1) As of right now, 99% of the network runs a single client. Anything this client rejects does de facto become a rule change. According to your own numbers even, this is not true. 99% of the network runs a wide variety of rules and versions. Even with a critical security announcement, the percentage of those running the latest version is not large. 2) Even if there were a diverse ecosystem of clients in place, discouragement rules that potentially affect legitimate miners significantly mess with the odds of finding a block. 3) If legitimate miners do not adopt counter-rules to bypass these new restrictions, the illegitimate miners are left with an even larger percentage of blocks found. Miners are not the -only- ones that get a say in what is spam, and what is not. If miners are generating garbage, network users have the right to veto that garbage. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
On Thursday, May 24, 2012 4:33:12 PM Jeff Garzik wrote: Comments? It wouldn't be a problem if these no-TX blocks were not already getting frequent (1 in 20). FWIW, based on statistics for Eligius's past 100 blocks, it seems 10% (1 in 10) of 1-txn blocks is not actually unreasonable. This also means these 1-txn mined blocks are not necessarily harming Bitcoin intentionally. Anyone care to figure out the math for how fast miners need to finish processing transactions to reduce the number of 1txn blocks? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
On Thu, May 24, 2012 at 8:45 PM, Luke-Jr l...@dashjr.org wrote: On Thursday, May 24, 2012 4:33:12 PM Jeff Garzik wrote: Comments? It wouldn't be a problem if these no-TX blocks were not already getting frequent (1 in 20). FWIW, based on statistics for Eligius's past 100 blocks, it seems 10% (1 in 10) of 1-txn blocks is not actually unreasonable. This also means these 1-txn mined blocks are not necessarily harming Bitcoin intentionally. Anyone care to figure out the math for how fast miners need to finish processing transactions to reduce the number of 1txn blocks? Look at the time since last block, and correlate with the number of non-spam TX's in the memory pool at the time. It is obvious which ones are quick blocks (60 seconds since last block, no big deal) and which ones are the lazy miners ( 120 seconds since last block). -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Punishing empty blocks?
On Friday, May 25, 2012 12:51:09 AM Jeff Garzik wrote: On Thu, May 24, 2012 at 8:45 PM, Luke-Jr l...@dashjr.org wrote: On Thursday, May 24, 2012 4:33:12 PM Jeff Garzik wrote: Comments? It wouldn't be a problem if these no-TX blocks were not already getting frequent (1 in 20). FWIW, based on statistics for Eligius's past 100 blocks, it seems 10% (1 in 10) of 1-txn blocks is not actually unreasonable. This also means these 1-txn mined blocks are not necessarily harming Bitcoin intentionally. Anyone care to figure out the math for how fast miners need to finish processing transactions to reduce the number of 1txn blocks? Look at the time since last block, and correlate with the number of non-spam TX's in the memory pool at the time. It is obvious which ones are quick blocks (60 seconds since last block, no big deal) and which ones are the lazy miners ( 120 seconds since last block). Block times are not accurate enough for that. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
