Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?
On Mon, Jan 29, 2018 at 10:54 PM, Gregory Maxwell via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > On Mon, Jan 29, 2018 at 9:40 PM, Tier Nolan via bitcoin-dev > wrote: > > if there were tighter time requirements in the protocol > miners would address them by running NTP which as an _astounding_ lack > of security in terms of how it is commonly deployed. > Could you say a few more words about this lack of security? Or share a link if you have one. I know very little about NTPs. > ___ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?
On Mon, Jan 29, 2018 at 10:40 PM, Tier Nolan via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > > Much of Bitcoin operates on the assumption that a majority of miners are > honest. If 50%+ of miners set their timestamp reasonably accurately (say > within 10 mins), then the actual timestamp will move forward at the same > rate as real time. > Thank you for replying. I agree that under the 50%+ assumption, timestamps are reasonably accurately, but I fail to see a reason to make this assumption. I'm comfortable with the 50%+ assumption regarding ledger manipulation (double-spending, deletion of transactions etc.). I'm much less comfortable with it regarding timestamps manipulation. Consider the following situation: (1) miners are selfish, (2) miners have a financial incentive to be dishonest. (1) is a common state on how miners function nowadays. (2) is the case that interests us when coming to do this analysis. In the case of ledger manipulation, the 50%+ assumption is not because we assume that miners are good-hearted (this violates (1)). It is there due to an assumption that the financial damage to a miner would be bigger than the gain in (2). This happens since a ledge manipulation may cause miners to lose block rewards, and certainly will devaluate Bitcoin, an asset which they possess. In the case of timestamps manipulation, I don't see any financial damage caused to miners. Timestamps manipulation (besides the 2016*n blocks) won't harm the function of Bitcoin, and may even go undetected (it seems to me that the main blockchain explorers don't track it). I don't see a justification for the 50%+ assumption here. > > Dishonest miners could set their timestamp as low as possible, but the > median would move foward if more than half of the timestamps move forward. > > >> If we want to be pedantic, the best lower bound for a block timestamp is >> the timestamp of the block that closes the adjustment interval in which it >> resides. >> > > If you are assuming that the miners are majority dishonest, then they can > set the limit to anything as long as they don't move it more than 2 hours > into the future. > > The miners could set their timestamps so that they increase 1 week fake > time every 2 weeks real time and reject any blocks more than 2 hours ahead > of their fake time. The difficulty would settle so that one block occurs > every 20 mins. > > >> >> Possible improvement: >> - >> We may consider exchanging average with standard deviation in the >> difficulty adjustment formula. It both better mirrors changes in the hash >> power along the interval, and disables the option to manipulate timestamps >> without affecting the difficulty. >> >> I'm aware that this change requires a hardfork, and won't happen any time >> soon. But does it make sense to add it to a potential future hard fork? >> > > For check locktime, the median of the last 11 blocks is used as an > improved indicator of what the actual real time is. Again, it assumes that > a majority of the miners are honest. > >> >> >> ___ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> > > ___ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?
On Mon, Jan 29, 2018 at 09:54:23PM +, Gregory Maxwell via bitcoin-dev wrote: > If times need to be accurate Bitcoin would need to use a rather > different design (e.g. each block would commit to the observation time > of the prior N blocks, and an iterative algorithm would solve for each > blocks time and each miners local offset). > > IIRC open-timestamp calendar servers provide more precise > time-stamping under the assumption that the calendar server is > behaving correctly. That is incorrect. The OpenTimestamps servers are specifically designed not to be trusted, and thus do not make any cryptographically verifiable attestations as to when timestamps were created. In the future I expect to add a trusted timestamping scheme via disposable keys to the OpenTimestamps protocol, but that work isn't yet complete: https://lists.opentimestamps.org/pipermail/ots-dev/2017-May/01.html -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?
The terms "simple ordering of blocks" and timestamp are essentially the same thing. On Jan 29, 2018 1:16 PM, "Neiman via bitcoin-dev" < bitcoin-dev@lists.linuxfoundation.org> wrote: > First time posting here, please be gentle. > > I'm doing a research project about blockchain timestamping. There are many > such projects, including the fantastic OpenTimestamps. > > All of the projects essentially save some data in a block, and rely on the > block timestamp as a proof that this data existed at some specific time. > > But how accurate are Bitcoins timestamps? > > I didn't find any discussion or research regarding Bitcoin timestamp > accuracy (also not in the history of this mailing list). I share here a > simple analysis of timestamp accuracy, and a suggestion how to improve it. > > Basic observations and questions: > --- > *1.* It seems to me that the timestamp is not the time that the block was > created. Ideally, it's the time that the miner started to try to mine the > block. However, as timestamps may also be used as a source of variety for > hashes, the exact meaning of its value is unclear. > > If this is true, then there's a strange phenomena to observe in > blockchain.info and blockexplorer.com: the timestamps of blocks equals > the receiving times. > > Am I wrong in my understanding, or is there a mistake in those websites? > > *2.* Timestamps are not necessary to avoid double-spending. A simple > ordering of blocks is sufficient, so exchanging timestamps with enumeration > would work double-spending wise. Permissioned consensus protocols, such as > hyperledger, indeed have no timestamps (in version 1.0). > > As far as I could tell, timestamps are included in Bitcoin's protocol > *only* to adjust the difficulty of PoW. > > Direct control of timestamp accuracy: > --- > The only element in the protocol that I found to control timestamp > accuracy is based on the network time concept. > > The Bitcoin protocol defines “network time” for each node. The network > time is the median time of the other clients, but only if > 1. there are at least 5 connected, and > 2. the difference between the median time and the nodes own system > time is less than 70 minutes. > > Then new blocks are accepted by the peers if their timestamps is > 1. less than the network time plus 2 hours, and > 2. greater than the median timestamp of previous 11 blocks. > > The first rule supplies a 2 hour upper bound for timestamp accuracy. > > However, the second rule doesn't give a tight lower bound. Actually, no > lower bound is given at all if no assumption is made about the median. If > we assume the median to be accurate enough at some timepoint, then we're > only assured that any future timestamp is no bigger than this specific > median, which is not much information. > > Further analysis can be made under different assumptions. For example, > what's the accuracy if holders of 51% of the computational power create > honest timestamps? But unfortunately, I don't see any good reason to work > under such an assumptions. > > The second rule cannot be strengthened to be similar to the first one > (i.e., nodes don't accept blocks less than network time minus 2 hours). The > reason is that nodes cannot differentiate if it's a new block with > dishonest timestamp, an old block with an old timestamps (with many other > blocks coming) or simply a new block that took a long time to mine. > > Indirect control of timestamps accuracy: > -- > If we assume that miners have no motive to increase difficulty > artificially, then the PoW adjusting algorithm yields a second mechanism of > accuracy control. > > The adjustment rules are given in pow.cpp (bitcoin-core source, version > 0.15.1), in the function 'CalculateNextWorkRequired', by the formula (with > some additional adjustments which I omit): > > (old_target* (time_of_last_block_in_2016_blocks_interval - > time_of_first_block_in_2016_blocks_interval) )/time_of_two_weeks > > It uses a simple average of block time in the last 2016 blocks. But such > averages ignore any values besides the first and last one in the interval. > Hence, if the difficulty is constant, the following sequence is valid from > both the protocol and the miners incentives point of views: > > 1, 2, 3,…., 2015, 1209600 (time of two weeks), 2017, 2018, 2019,…., > 4031, 1209600*2, 4033, 4044, … > > If we want to be pedantic, the best lower bound for a block timestamp is > the timestamp of the block that closes the adjustment interval in which it > resides. > > Possible improvement: > - > We may consider exchanging average with standard deviation in the > difficulty adjustment formula. It both better mirrors changes in the hash > power along the interval, and disables the option to manipulate timestamps > without affecting the difficulty. > > I'm
Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?
On Mon, Jan 29, 2018 at 7:34 AM, Neiman via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > > But how accurate are Bitcoins timestamps? > A perspective on block timestamp and opentimestamps can be found here: https://lists.w3.org/Archives/Public/public-blockchain/2016Sep/0076.html - Bryan http://heybryan.org/ 1 512 203 0507 ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?
On Mon, Jan 29, 2018 at 9:40 PM, Tier Nolan via bitcoin-dev wrote: > For check locktime, the median of the last 11 blocks is used as an improved > indicator of what the actual real time is. Again, it assumes that a > majority of the miners are honest. It would be more accurate to say that the median is not used for improved accuracy but to mitigate a consensus incompatibility: If the block's own timestamp were used for nlocktime and time based nlocks were common on the network each miner would maximize their fee income by setting the value as high as they could get away with. What concerned us wasn't so much that this would make the times less accurate (though it would) but rather that it would create an incentive for a runaway situation that could harm network stability (e.g. with all miners cranking times against the 2hr window, then creating pressure for miners to accept further and further in the future; each responding to his own local incentives). This incentive incompatibility could have been addressed e.g. by using the prior block's time, but since the protocol doesn't require times to be monotone (and for good reason!) the simple implementation of that wouldn't have been a soft-fork. The 11 block MTP worked out nicely because the protocol already required new times to be larger than that. The timestamps in Bitcoin aren't intended to be particularly accurate. They're used only for controlling the difficulty, and the adjustment window is large enough that there isn't much distortion that can be accomplished there. It's not clear to me that much better can really be done... if there were tighter time requirements in the protocol miners would address them by running NTP which as an _astounding_ lack of security in terms of how it is commonly deployed. As far as I know, I'm the only person whos ever mined blocks with their own stratum 1 time source. If times need to be accurate Bitcoin would need to use a rather different design (e.g. each block would commit to the observation time of the prior N blocks, and an iterative algorithm would solve for each blocks time and each miners local offset). IIRC open-timestamp calendar servers provide more precise time-stamping under the assumption that the calendar server is behaving correctly. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?
On Mon, Jan 29, 2018 at 1:34 PM, Neiman via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > > *2.* Timestamps are not necessary to avoid double-spending. A simple > ordering of blocks is sufficient, so exchanging timestamps with enumeration > would work double-spending wise. Permissioned consensus protocols, such as > hyperledger, indeed have no timestamps (in version 1.0). > The timestamps simply needs to be reasonably accurate. Their main purpose is to allow difficulty updates. They can also be used to check that the node has caught up. > It uses a simple average of block time in the last 2016 blocks. But such > averages ignore any values besides the first and last one in the interval. > Hence, if the difficulty is constant, the following sequence is valid from > both the protocol and the miners incentives point of views: > > 1, 2, 3,…., 2015, 1209600 (time of two weeks), 2017, 2018, 2019,…., > 4031, 1209600*2, 4033, 4044, … > Much of Bitcoin operates on the assumption that a majority of miners are honest. If 50%+ of miners set their timestamp reasonably accurately (say within 10 mins), then the actual timestamp will move forward at the same rate as real time. Dishonest miners could set their timestamp as low as possible, but the median would move foward if more than half of the timestamps move forward. > If we want to be pedantic, the best lower bound for a block timestamp is > the timestamp of the block that closes the adjustment interval in which it > resides. > If you are assuming that the miners are majority dishonest, then they can set the limit to anything as long as they don't move it more than 2 hours into the future. The miners could set their timestamps so that they increase 1 week fake time every 2 weeks real time and reject any blocks more than 2 hours ahead of their fake time. The difficulty would settle so that one block occurs every 20 mins. > > Possible improvement: > - > We may consider exchanging average with standard deviation in the > difficulty adjustment formula. It both better mirrors changes in the hash > power along the interval, and disables the option to manipulate timestamps > without affecting the difficulty. > > I'm aware that this change requires a hardfork, and won't happen any time > soon. But does it make sense to add it to a potential future hard fork? > For check locktime, the median of the last 11 blocks is used as an improved indicator of what the actual real time is. Again, it assumes that a majority of the miners are honest. > > > ___ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[bitcoin-dev] How accurate are the Bitcoin timestamps?
First time posting here, please be gentle. I'm doing a research project about blockchain timestamping. There are many such projects, including the fantastic OpenTimestamps. All of the projects essentially save some data in a block, and rely on the block timestamp as a proof that this data existed at some specific time. But how accurate are Bitcoins timestamps? I didn't find any discussion or research regarding Bitcoin timestamp accuracy (also not in the history of this mailing list). I share here a simple analysis of timestamp accuracy, and a suggestion how to improve it. Basic observations and questions: --- *1.* It seems to me that the timestamp is not the time that the block was created. Ideally, it's the time that the miner started to try to mine the block. However, as timestamps may also be used as a source of variety for hashes, the exact meaning of its value is unclear. If this is true, then there's a strange phenomena to observe in blockchain.info and blockexplorer.com: the timestamps of blocks equals the receiving times. Am I wrong in my understanding, or is there a mistake in those websites? *2.* Timestamps are not necessary to avoid double-spending. A simple ordering of blocks is sufficient, so exchanging timestamps with enumeration would work double-spending wise. Permissioned consensus protocols, such as hyperledger, indeed have no timestamps (in version 1.0). As far as I could tell, timestamps are included in Bitcoin's protocol *only* to adjust the difficulty of PoW. Direct control of timestamp accuracy: --- The only element in the protocol that I found to control timestamp accuracy is based on the network time concept. The Bitcoin protocol defines “network time” for each node. The network time is the median time of the other clients, but only if 1. there are at least 5 connected, and 2. the difference between the median time and the nodes own system time is less than 70 minutes. Then new blocks are accepted by the peers if their timestamps is 1. less than the network time plus 2 hours, and 2. greater than the median timestamp of previous 11 blocks. The first rule supplies a 2 hour upper bound for timestamp accuracy. However, the second rule doesn't give a tight lower bound. Actually, no lower bound is given at all if no assumption is made about the median. If we assume the median to be accurate enough at some timepoint, then we're only assured that any future timestamp is no bigger than this specific median, which is not much information. Further analysis can be made under different assumptions. For example, what's the accuracy if holders of 51% of the computational power create honest timestamps? But unfortunately, I don't see any good reason to work under such an assumptions. The second rule cannot be strengthened to be similar to the first one (i.e., nodes don't accept blocks less than network time minus 2 hours). The reason is that nodes cannot differentiate if it's a new block with dishonest timestamp, an old block with an old timestamps (with many other blocks coming) or simply a new block that took a long time to mine. Indirect control of timestamps accuracy: -- If we assume that miners have no motive to increase difficulty artificially, then the PoW adjusting algorithm yields a second mechanism of accuracy control. The adjustment rules are given in pow.cpp (bitcoin-core source, version 0.15.1), in the function 'CalculateNextWorkRequired', by the formula (with some additional adjustments which I omit): (old_target* (time_of_last_block_in_2016_blocks_interval - time_of_first_block_in_2016_blocks_interval) )/time_of_two_weeks It uses a simple average of block time in the last 2016 blocks. But such averages ignore any values besides the first and last one in the interval. Hence, if the difficulty is constant, the following sequence is valid from both the protocol and the miners incentives point of views: 1, 2, 3,…., 2015, 1209600 (time of two weeks), 2017, 2018, 2019,…., 4031, 1209600*2, 4033, 4044, … If we want to be pedantic, the best lower bound for a block timestamp is the timestamp of the block that closes the adjustment interval in which it resides. Possible improvement: - We may consider exchanging average with standard deviation in the difficulty adjustment formula. It both better mirrors changes in the hash power along the interval, and disables the option to manipulate timestamps without affecting the difficulty. I'm aware that this change requires a hardfork, and won't happen any time soon. But does it make sense to add it to a potential future hard fork? ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev