Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?

2018-01-30 Thread Neiman via bitcoin-dev
On Mon, Jan 29, 2018 at 10:54 PM, Gregory Maxwell via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Mon, Jan 29, 2018 at 9:40 PM, Tier Nolan via bitcoin-dev
>  wrote:
>
>  if there were tighter time requirements in the protocol
> miners would address them by running NTP which as an _astounding_ lack
> of security in terms of how it is commonly deployed.
>

Could you say a few more words about this lack of security? Or share a link
if you have one. I know very little about NTPs.


> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?

2018-01-30 Thread Neiman via bitcoin-dev
On Mon, Jan 29, 2018 at 10:40 PM, Tier Nolan via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

>
> Much of Bitcoin operates on the assumption that a majority of miners are
> honest.  If 50%+ of miners set their timestamp reasonably accurately (say
> within 10 mins), then the actual timestamp will move forward at the same
> rate as real time.
>

Thank you for replying. I agree that under the 50%+ assumption, timestamps
are reasonably accurately, but I fail to see a reason to make this
assumption.

I'm comfortable with the 50%+ assumption regarding ledger manipulation
(double-spending, deletion of transactions etc.). I'm much less comfortable
with it regarding timestamps manipulation.

Consider the following situation:
(1) miners are selfish,
(2) miners have a financial incentive to be dishonest.

(1) is a common state on how miners function nowadays. (2) is the case that
interests us when coming to do this analysis.

In the case of ledger manipulation, the 50%+ assumption is not because we
assume that miners are good-hearted (this violates (1)). It is there due to
an assumption that the financial damage to a miner would be bigger than the
gain in (2). This happens since a ledge manipulation may cause miners to
lose block rewards, and certainly will devaluate Bitcoin, an asset which
they possess.

In the case of timestamps manipulation, I don't see any financial damage
caused to miners. Timestamps manipulation (besides the 2016*n blocks) won't
harm the function of Bitcoin, and may even go undetected (it seems to me
that the main blockchain explorers don't track it). I don't see a
justification for the 50%+ assumption here.


>
> Dishonest miners could set their timestamp as low as possible, but the
> median would move foward if more than half of the timestamps move forward.
>
>
>> If we want to be pedantic, the best lower bound for a block timestamp is
>> the timestamp of the block that closes the adjustment interval in which it
>> resides.
>>
>
> If you are assuming that the miners are majority dishonest, then they can
> set the limit to anything as long as they don't move it more than 2 hours
> into the future.
>
> The miners could set their timestamps so that they increase 1 week fake
> time every 2 weeks real time and reject any blocks more than 2 hours ahead
> of their fake time.  The difficulty would settle so that one block occurs
> every 20 mins.
>
>
>>
>> Possible improvement:
>> -
>> We may consider exchanging average with standard deviation in the
>> difficulty adjustment formula. It both better mirrors changes in the hash
>> power along the interval, and disables the option to manipulate timestamps
>> without affecting the difficulty.
>>
>> I'm aware that this change requires a hardfork, and won't happen any time
>> soon. But does it make sense to add it to a potential future hard fork?
>>
>
> For check locktime, the median of the last 11 blocks is used as an
> improved indicator of what the actual real time is.  Again, it assumes that
> a majority of the miners are honest.
>
>>
>>
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?

2018-01-29 Thread George Balch via bitcoin-dev
The terms "simple ordering of blocks" and timestamp are essentially the
same thing.

On Jan 29, 2018 1:16 PM, "Neiman via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> First time posting here, please be gentle.
>
> I'm doing a research project about blockchain timestamping. There are many
> such projects, including the fantastic OpenTimestamps.
>
> All of the projects essentially save some data in a block, and rely on the
> block timestamp as a proof that this data existed at some specific time.
>
> But how accurate are Bitcoins timestamps?
>
> I didn't find any discussion or research regarding Bitcoin timestamp
> accuracy (also not in the history of this mailing list). I share here a
> simple analysis of timestamp accuracy, and a suggestion how to improve it.
>
> Basic observations and questions:
> ---
> *1.* It seems to me that the timestamp is not the time that the block was
> created. Ideally, it's the time that the miner started to try to mine the
> block. However, as timestamps may also be used as a source of variety for
> hashes, the exact meaning of its value is unclear.
>
> If this is true, then there's a strange phenomena to observe in
> blockchain.info and blockexplorer.com: the timestamps of blocks equals
> the receiving times.
>
> Am I wrong in my understanding, or is there a mistake in those websites?
>
> *2.* Timestamps are not necessary to avoid double-spending. A simple
> ordering of blocks is sufficient, so exchanging timestamps with enumeration
> would work double-spending wise. Permissioned consensus protocols, such as
> hyperledger, indeed have no timestamps (in version 1.0).
>
> As far as I could tell, timestamps are included in Bitcoin's protocol
> *only* to adjust the difficulty of PoW.
>
> Direct control of timestamp accuracy:
> ---
> The only element in the protocol that I found to control timestamp
> accuracy is based on the network time concept.
>
> The Bitcoin protocol defines “network time” for each node. The network
> time is the median time of the other clients, but only if
> 1. there are at least 5 connected, and
> 2. the difference between the median time and the nodes own system
> time is less than 70 minutes.
>
> Then new blocks are accepted by the peers if their timestamps is
> 1. less than the network time plus 2 hours, and
> 2. greater than the median timestamp of previous 11 blocks.
>
> The first rule supplies a 2 hour upper bound for timestamp accuracy.
>
> However, the second rule doesn't give a tight lower bound. Actually, no
> lower bound is given at all if no assumption is made about the median. If
> we assume the median to be accurate enough at some timepoint, then we're
> only assured that any future timestamp is no bigger than this specific
> median, which is not much information.
>
> Further analysis can be made under different assumptions. For example,
> what's the accuracy if holders of 51% of the computational power create
> honest timestamps? But unfortunately, I don't see any good reason to work
> under such an assumptions.
>
> The second rule cannot be strengthened to be similar to the first one
> (i.e., nodes don't accept blocks less than network time minus 2 hours). The
> reason is that nodes cannot differentiate if it's a new block with
> dishonest timestamp, an old block with an old timestamps (with many other
> blocks coming) or simply a new block that took a long time to mine.
>
> Indirect control of timestamps accuracy:
> --
> If we assume that miners have no motive to increase difficulty
> artificially, then the PoW adjusting algorithm yields a second mechanism of
> accuracy control.
>
> The adjustment rules are given in pow.cpp (bitcoin-core source, version
> 0.15.1), in the function 'CalculateNextWorkRequired', by the formula (with
> some additional adjustments which I omit):
>
> (old_target* (time_of_last_block_in_2016_blocks_interval -
> time_of_first_block_in_2016_blocks_interval) )/time_of_two_weeks
>
> It uses a simple average of block time in the last 2016 blocks. But such
> averages ignore any values besides the first and last one in the interval.
> Hence, if the difficulty is constant, the following sequence is valid from
> both the protocol and the miners incentives point of views:
>
> 1, 2, 3,…., 2015, 1209600 (time of two weeks), 2017, 2018, 2019,….,
> 4031, 1209600*2, 4033, 4044, …
>
> If we want to be pedantic, the best lower bound for a block timestamp is
> the timestamp of the block that closes the adjustment interval in which it
> resides.
>
> Possible improvement:
> -
> We may consider exchanging average with standard deviation in the
> difficulty adjustment formula. It both better mirrors changes in the hash
> power along the interval, and disables the option to manipulate timestamps
> without affecting the difficulty.
>
> 

Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?

2018-01-29 Thread Bryan Bishop via bitcoin-dev
On Mon, Jan 29, 2018 at 7:34 AM, Neiman via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> But how accurate are Bitcoins timestamps?
>

A perspective on block timestamp and opentimestamps can be found here:
https://lists.w3.org/Archives/Public/public-blockchain/2016Sep/0076.html

- Bryan
http://heybryan.org/
1 512 203 0507
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?

2018-01-29 Thread Gregory Maxwell via bitcoin-dev
On Mon, Jan 29, 2018 at 9:40 PM, Tier Nolan via bitcoin-dev
 wrote:
> For check locktime, the median of the last 11 blocks is used as an improved
> indicator of what the actual real time is.  Again, it assumes that a
> majority of the miners are honest.

It would be more accurate to say that the median is not used for
improved accuracy but to mitigate a consensus incompatibility:

If the block's own timestamp were used for nlocktime and time based
nlocks were common on the network each miner would maximize their fee
income by setting the value as high as they could get away with.  What
concerned us wasn't so much that this would make the times less
accurate (though it would) but rather that it would create an
incentive for a runaway situation that could harm network stability
(e.g. with all miners cranking times against the 2hr window, then
creating pressure for miners to accept further and further in the
future; each responding to his own local incentives).

This incentive incompatibility could have been addressed e.g. by using
the prior block's time, but since the protocol doesn't require times
to be monotone (and for good reason!) the simple implementation of
that wouldn't have been a soft-fork.  The 11 block MTP worked out
nicely because the protocol already required new times to be larger
than that.

The timestamps in Bitcoin aren't intended to be particularly accurate.
They're used only for controlling the difficulty, and the adjustment
window is large enough that there isn't much distortion that can be
accomplished there.  It's not clear to me that much better can really
be done... if there were tighter time requirements in the protocol
miners would address them by running NTP which as an _astounding_ lack
of security in terms of how it is commonly deployed.  As far as I
know, I'm the only person whos ever mined blocks with their own
stratum 1 time source.

If times need to be accurate Bitcoin would need to use a rather
different design (e.g. each block would commit to the observation time
of the prior N blocks, and an iterative algorithm would solve for each
blocks time and each miners local offset).

IIRC open-timestamp calendar servers provide more precise
time-stamping under the assumption that the calendar server is
behaving correctly.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


Re: [bitcoin-dev] How accurate are the Bitcoin timestamps?

2018-01-29 Thread Tier Nolan via bitcoin-dev
On Mon, Jan 29, 2018 at 1:34 PM, Neiman via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

>
> *2.* Timestamps are not necessary to avoid double-spending. A simple
> ordering of blocks is sufficient, so exchanging timestamps with enumeration
> would work double-spending wise. Permissioned consensus protocols, such as
> hyperledger, indeed have no timestamps (in version 1.0).
>

The timestamps simply needs to be reasonably accurate.  Their main purpose
is to allow difficulty updates.

They can also be used to check that the node has caught up.


> It uses a simple average of block time in the last 2016 blocks. But such
> averages ignore any values besides the first and last one in the interval.
> Hence, if the difficulty is constant, the following sequence is valid from
> both the protocol and the miners incentives point of views:
>
> 1, 2, 3,…., 2015, 1209600 (time of two weeks), 2017, 2018, 2019,….,
> 4031, 1209600*2, 4033, 4044, …
>

Much of Bitcoin operates on the assumption that a majority of miners are
honest.  If 50%+ of miners set their timestamp reasonably accurately (say
within 10 mins), then the actual timestamp will move forward at the same
rate as real time.

Dishonest miners could set their timestamp as low as possible, but the
median would move foward if more than half of the timestamps move forward.


> If we want to be pedantic, the best lower bound for a block timestamp is
> the timestamp of the block that closes the adjustment interval in which it
> resides.
>

If you are assuming that the miners are majority dishonest, then they can
set the limit to anything as long as they don't move it more than 2 hours
into the future.

The miners could set their timestamps so that they increase 1 week fake
time every 2 weeks real time and reject any blocks more than 2 hours ahead
of their fake time.  The difficulty would settle so that one block occurs
every 20 mins.


>
> Possible improvement:
> -
> We may consider exchanging average with standard deviation in the
> difficulty adjustment formula. It both better mirrors changes in the hash
> power along the interval, and disables the option to manipulate timestamps
> without affecting the difficulty.
>
> I'm aware that this change requires a hardfork, and won't happen any time
> soon. But does it make sense to add it to a potential future hard fork?
>

For check locktime, the median of the last 11 blocks is used as an improved
indicator of what the actual real time is.  Again, it assumes that a
majority of the miners are honest.

>
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


[bitcoin-dev] How accurate are the Bitcoin timestamps?

2018-01-29 Thread Neiman via bitcoin-dev
First time posting here, please be gentle.

I'm doing a research project about blockchain timestamping. There are many
such projects, including the fantastic OpenTimestamps.

All of the projects essentially save some data in a block, and rely on the
block timestamp as a proof that this data existed at some specific time.

But how accurate are Bitcoins timestamps?

I didn't find any discussion or research regarding Bitcoin timestamp
accuracy (also not in the history of this mailing list). I share here a
simple analysis of timestamp accuracy, and a suggestion how to improve it.

Basic observations and questions:
---
*1.* It seems to me that the timestamp is not the time that the block was
created. Ideally, it's the time that the miner started to try to mine the
block. However, as timestamps may also be used as a source of variety for
hashes, the exact meaning of its value is unclear.

If this is true, then there's a strange phenomena to observe in
blockchain.info and blockexplorer.com: the timestamps of blocks equals the
receiving times.

Am I wrong in my understanding, or is there a mistake in those websites?

*2.* Timestamps are not necessary to avoid double-spending. A simple
ordering of blocks is sufficient, so exchanging timestamps with enumeration
would work double-spending wise. Permissioned consensus protocols, such as
hyperledger, indeed have no timestamps (in version 1.0).

As far as I could tell, timestamps are included in Bitcoin's protocol
*only* to adjust the difficulty of PoW.

Direct control of timestamp accuracy:
---
The only element in the protocol that I found to control timestamp accuracy
is based on the network time concept.

The Bitcoin protocol defines “network time” for each node. The network time
is the median time of the other clients, but only if
1. there are at least 5 connected, and
2. the difference between the median time and the nodes own system time
is less than 70 minutes.

Then new blocks are accepted by the peers if their timestamps is
1. less than the network time plus 2 hours, and
2. greater than the median timestamp of previous 11 blocks.

The first rule supplies a 2 hour upper bound for timestamp accuracy.

However, the second rule doesn't give a tight lower bound. Actually, no
lower bound is given at all if no assumption is made about the median. If
we assume the median to be accurate enough at some timepoint, then we're
only assured that any future timestamp is no bigger than this specific
median, which is not much information.

Further analysis can be made under different assumptions. For example,
what's the accuracy if holders of 51% of the computational power create
honest timestamps? But unfortunately, I don't see any good reason to work
under such an assumptions.

The second rule cannot be strengthened to be similar to the first one
(i.e., nodes don't accept blocks less than network time minus 2 hours). The
reason is that nodes cannot differentiate if it's a new block with
dishonest timestamp, an old block with an old timestamps (with many other
blocks coming) or simply a new block that took a long time to mine.

Indirect control of timestamps accuracy:
--
If we assume that miners have no motive to increase difficulty
artificially, then the PoW adjusting algorithm yields a second mechanism of
accuracy control.

The adjustment rules are given in pow.cpp (bitcoin-core source, version
0.15.1), in the function 'CalculateNextWorkRequired', by the formula (with
some additional adjustments which I omit):

(old_target* (time_of_last_block_in_2016_blocks_interval -
time_of_first_block_in_2016_blocks_interval) )/time_of_two_weeks

It uses a simple average of block time in the last 2016 blocks. But such
averages ignore any values besides the first and last one in the interval.
Hence, if the difficulty is constant, the following sequence is valid from
both the protocol and the miners incentives point of views:

1, 2, 3,…., 2015, 1209600 (time of two weeks), 2017, 2018, 2019,….,
4031, 1209600*2, 4033, 4044, …

If we want to be pedantic, the best lower bound for a block timestamp is
the timestamp of the block that closes the adjustment interval in which it
resides.

Possible improvement:
-
We may consider exchanging average with standard deviation in the
difficulty adjustment formula. It both better mirrors changes in the hash
power along the interval, and disables the option to manipulate timestamps
without affecting the difficulty.

I'm aware that this change requires a hardfork, and won't happen any time
soon. But does it make sense to add it to a potential future hard fork?
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev