[Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread Addy Yeow
Hi list,

Can someone explain why do we have 32-bit and 64-bit timestamp fields
instead of all being 64-bit?

https://en.bitcoin.it/wiki/Protocol_specification

Cheers,
Addy

--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread Jeff Garzik
On Wed, May 8, 2013 at 7:39 PM, Addy Yeow ayeo...@gmail.com wrote:
 Hi list,

 Can someone explain why do we have 32-bit and 64-bit timestamp fields
 instead of all being 64-bit?

 https://en.bitcoin.it/wiki/Protocol_specification

Hysterical raisins.

-- 
Jeff Garzik
exMULTI, Inc.
jgar...@exmulti.com

--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread Peter Todd
On Thu, May 09, 2013 at 09:39:10AM +1000, Addy Yeow wrote:
 Hi list,
 
 Can someone explain why do we have 32-bit and 64-bit timestamp fields
 instead of all being 64-bit?
 
 https://en.bitcoin.it/wiki/Protocol_specification

Who knows?

Satoshi used 32-bits and those fields can't be changed now without every
single Bitcoin user changing all at once. (a hard-fork change)

We'll probably need to do one of those eventually for other reasons, so
we might as well leave fixing the timestamps until then.

-- 
'peter'[:-1]@petertodd.org
002a9a85a940c4da2951c3e91a043a44805fa286b336364d9daa


signature.asc
Description: Digital signature
--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Discovery/addr packets (was: Service bits for pruned nodes)

2013-05-08 Thread John Dillon
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

  You mean scam you with a zero-conf transaction that hasn't actually been
  broadcast?

 Yeah. Or just scam you at all. It's hard to imagine an organisation as
 a big as a mobile carrier engaging in financial scamming (roaming fees
 excepted).

Unless the government told them too.

 I've said this before, but I think it's worth repeating. The
 double-spend protection the block chain gives you has a sweet spot
 where it's really, really valuable (essential even) and then there are
 lots of kinds of transactions on either side of that sweet spot that
 don't really benefit from it.

 Obvious/trivial case where you don't need a block chain - Facebook
 buys Instagram for a gajillion coins. The legal system is plenty good
 enough to ensure the payments are honoured. Another example, when my
 employer pays me my salary. They aren't going to double spend this
 except through some horrible accident that we can get sorted out some
 other way.

The employer example actually shows something important: between a worker and
an employer double-spending already irrelevant. People get paid after they work
their two weeks not before, so the double-spend is already irrelevant.

However when your employer pays you on the blockchain until the transaction
confirms for someone else to accept funds from that payment they not only have
to trust you, but also the employer. Sure they could take it as you said you
would apy me so it is your responsibility to make that happen but that brings
a whole new level of complexity.

A scheme where you vouch for your payments with your identity can benifit from
being able to follow that chain all the way back to the last confirmed
transaction, although actually implementing this may be too complex to be
worthwhile, especially initially.

 Another case, very small payments. This is Satoshi's bag of crisps
 example. If the cost/complexity of double spending is higher than what
 the payment is worth, again, you don't really need the block chain.
 That's why it's worth optimising unconfirmed transactions to be harder
 to double spend, it optimises (pushes up) that lower bar.

Yes. But the issue is how are you going to optmize it? By adding yet more
restrictions and limitations on those who chose to run a node or mining
operation, or by actually fixing the trust issue? We know you can do the
latter, so do not sacrifice Bitcoin's core layer in silly attempts to make
double-spends harder. Fundementally Bitcoin has exactly one way of achieving
consensus, and that is the blockchain.

It must be your right to chose what transactins you chose to mine and chose to
relay. End of story. Bitcoin is not about imposing regulation on those who
choose to use it.

 Place where you really want the chain - largeish sums of money are
 moving around, but not large enough to justify expensive
 cross-jurisdictional legal action, or where the cost of identity
 verification and all the associated paperwork is just too high. I
 guess most online transactions fall into this bucket today.

Indeed. Especially for the most popular use of Bitcoin as a payment system:
buying things PayPal won't let you. In that circumstance the only leverage you
have is the protections of the blockchain and the damage you can do to the
other (often anonymous) parties reputation.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBCAAGBQJRivRcAAoJEEWCsU4mNhiPoJcIAL7T/x5gipsCNn/w3EfhZhKo
iukP0Kc3cni/Kb6gJrOlXufIxDrX8QxEhbIIrypFbyg+xHPK8NzSd13ScKNtLgjM
w2uOI/IkgUh7VLIEZADqLO3TM5S5VDZ/A42yzTIq8MeWxaTBD1JulOc/RbljGu8V
UrF6ptxu2UXTc0eXcor1lHfJRVteTJAAba5Awa1EAHX8f2c/1FhdrnOZwfLVJIfK
/nnUgqGKc8l08knC6NnAlP39zbk/FHiZF/keWIFIzhiyXTnqKnqD096tIx6MpPci
LGafjCoXACpr1XeiSufER/z6WxTvOvCbWRw4MYrbyRkmChqMtc8a7RMEPLXhaMQ=
=Oh/p
-END PGP SIGNATURE-

--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread John Dillon
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Wed, May 8, 2013 at 11:44 PM, Peter Todd p...@petertodd.org wrote:
 Who knows?

 Satoshi used 32-bits and those fields can't be changed now without every
 single Bitcoin user changing all at once. (a hard-fork change)

 We'll probably need to do one of those eventually for other reasons, so
 we might as well leave fixing the timestamps until then.

Perhaps Satoshi did this delibrately, knowing that at some point a hard-fork
would be a good idea, so that we all would have a good excuse to do one?
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBCAAGBQJRivT1AAoJEEWCsU4mNhiPhxUH/271jtMvNrliZxFTvud282Dc
snEieMig1p/HXy6ry1YLp+Q2k4Ya3QFFPlbsqHeTjL+NaJSGOHPBen4lpWahOH+T
N6TQoOls7BMpQ7Y54Nqy5Qh9GeQnbDGcbQ8fBUfFqAF1Ljs0OBsbJtvC3vZTbYEn
dwB+7dvPLGKVfz/yrR9wrLhDzoXHbG4C3sefqNnm+fkHHIuTy4nxwtVVMydlzerF
Bwg1oc64dlul7sugBGXo2FjtGrxxkRxWWqj+dPgBnE/bDKIlemw34PtQZ2OK+IUS
CH7Q0EGBnr7TpXJT5AOMkycd8v9MJ2wNIt4v3YLqyViQ48Q5coxAS0GepcRnbMU=
=na4H
-END PGP SIGNATURE-

--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread Jeff Garzik
On Wed, May 8, 2013 at 9:00 PM, John Dillon
john.dillon...@googlemail.com wrote:
 Perhaps Satoshi did this delibrately, knowing that at some point a hard-fork
 would be a good idea, so that we all would have a good excuse to do one?

Guffaw :)  The year 2038 is so far in the future that it is not really
relevant, from that angle.

We need a hard fork to break the 1MB limit, and Satoshi explicitly
presumed that would happen sometime in the future.

-- 
Jeff Garzik
exMULTI, Inc.
jgar...@exmulti.com

--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread Pieter Wuille
On Wed, May 08, 2013 at 09:08:34PM -0400, Jeff Garzik wrote:
 On Wed, May 8, 2013 at 9:00 PM, John Dillon
 john.dillon...@googlemail.com wrote:
  Perhaps Satoshi did this delibrately, knowing that at some point a hard-fork
  would be a good idea, so that we all would have a good excuse to do one?
 
 Guffaw :)  The year 2038 is so far in the future that it is not really
 relevant, from that angle.

Meh. I think it's highly unlikely we'll break the block header format, as it
pretty much means invalidating all mining hardware.

There's also no need: 32 bits is plenty of precision. Hell, even 16 bits would
do (assuming there's never more than a 65535s (about 18 hours) gap between two
blocks). Just assume the full 64-bit time is the smallest one that makes
sense, given its lower 32 bits.

-- 
Pieter


--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread John Dillon
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Thu, May 9, 2013 at 1:13 AM, Pieter Wuille pieter.wui...@gmail.com wrote:
 On Wed, May 08, 2013 at 09:08:34PM -0400, Jeff Garzik wrote:
 Guffaw :)  The year 2038 is so far in the future that it is not really
 relevant, from that angle.

 Meh. I think it's highly unlikely we'll break the block header format, as it
 pretty much means invalidating all mining hardware.

Doesn't most mining hardware at the ASCI level start with a SHA256 midstate
given that the nonce is at the end?  Adding further information to the block
should be possible at the beginning of the block without major changes to the
mining hardware.

 There's also no need: 32 bits is plenty of precision. Hell, even 16 bits would
 do (assuming there's never more than a 65535s (about 18 hours) gap between two
 blocks). Just assume the full 64-bit time is the smallest one that makes
 sense, given its lower 32 bits.

I feel somewhat uncomfortable about the after-the-fact auditing possible in
this scenario. Besides the timestamping provided by the block headers appears
to be useful in some payment protocols, not to mention in general.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBCAAGBQJRivnmAAoJEEWCsU
4mNhiPUIYH/AlxK4DHvIdq0khNH0nfK65E
F1ZyYZTGLNHKqrJLCU2kc7zteGadQuccmFsYpmViIr14tzpU7xMImUHpj7fEHO3R
S/1zy59rx2+VYcevYdwMDTywanjeForRpli93Hz570GfwfG/D7VPejfLo6iq5dOt
EG5m3Z8F7wNzWBmfBYBHKLrNBJe6iw0qJ2nNiHXcELt6gaqG3C9wI9NAPtQWQKjB
57h7yTnFCRmjA3HDdCe2s0FVJgRP5cJqz3e62qZrY/BRmw/Vrx8ExuX1LJFqUx3k
5tg+BxXH4DJbNIojuq9lLl5lWxKOI1iSJJuCAixo/6s/manLFggJv7KtYgzhhjI=
=BxDb
-END PGP SIGNATURE-

--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread Peter Todd
On Thu, May 09, 2013 at 01:27:33AM +, John Dillon wrote:
  There's also no need: 32 bits is plenty of precision. Hell, even 16 bits 
  would
  do (assuming there's never more than a 65535s (about 18 hours) gap between 
  two
  blocks). Just assume the full 64-bit time is the smallest one that makes
  sense, given its lower 32 bits.
 
 I feel somewhat uncomfortable about the after-the-fact auditing possible in
 this scenario. Besides the timestamping provided by the block headers appears
 to be useful in some payment protocols, not to mention in general.

Remember that interpreting the timestamp on a block for the purposes of
timestamping is a lot more subtle than it appears at first.

Any node will accept a block with a timestamp no more than two hours
ahead of what it thinks the current time is. That time is adjusted by
the median of the timestamps reported by your peers. For instance the
RPC call getinfo returns, among other things:

{
timeoffset : -1,
}

That is saying my node's wall clock time is 1 second behind the median
reported by it's peers - pretty good!


Naively you might think this means block timestamps are accurate to
within 2 hours right? Well, it's not so simple. Nodes will accept any
block with any timestamp *after* the median of the last 11 blocks. From
CBlock::AcceptBlock():

// Check timestamp against prev
if (GetBlockTime() = pindexPrev-GetMedianTimePast())
return state.Invalid(error(AcceptBlock() : block's timestamp is too 
early));

So in theory a miner could prevent that block from moving forward,
although if they do they drive up the difficulty, so all miners have an
incentive to set the timestamp accurately.

There are two types of timestamps possible: proofs that data existed
before a time, and proofs that data existed after. With the former type
the *later* the proof says the data existed, the more conservative the
assumptions behind the proof. So simply adding two hours to the block's
timestamp is relatively reasonable. (this assumes the attack managed to
mine a single block, and all nodes have accurate clocks)


The latter type, where you prove data existed after a given time, is a
much more tricky thing to apply. The genesis block is a great example
with that famous newspaper headline:

The Times 03/Jan/2009 Chancellor on brink of second bailout for
banks

As I mentioned in my other (private) email to you a few minutes ago, the
sig of my emails has the latest block hash in each one. The basic idea
is called a random beacon; NIST has a good description and a project to
create one:

http://www.nist.gov/itl/csd/ct/nist_beacon.cfm

Now technically speaking a random beacon is actually a more
sophisticated concept than just timestamping, the random beacon's value
is public and distributed widely, but for timestamping the idea is
basically to have an unpredictable number known to have been produced at
a certain time.

So you know this email was written after block #235235, timestamp
2013-05-09 01:21:52 right? Not so fast. All you actually know is the PGP
*signature* was created after that time, because the actual text of the
email is independent of the beacon nonce. (dunno if I have the correct
terminology here FWIW)

For a blockchain it's easy enough, the blocks naturally depend on a
genesis block, but applying the concept more generally is tricky and
application dependent; consider for example proving you created a
keypair after some data, which might be a useful thing to prove if the
secret key was created in some tamperproof hardware that you know has
left the factory and is in your possesion. It's easy to see how to do
this with ECC: just use the same techniques as in HD wallets to derive
keys.

To use the blockchain as a secure random beacon you need to make two
assumptions, 50% of the hashing power is controlled by honest miners,
and those honest miners have accurate clocks. With those assumptions you
can work out what is the minimum possible time the block could have been
accepted by the GetMedianTimePast() function and you are good to go.

What do people do in practice? Well look at
http://vog.github.io/bitcoinproof/, they just give the timestamp and
nothing else. Same for OpenTimestamps. (although I'm adding this email
to my notes, half the reason it's so detailed...)


Back to the block header time... Frankly, the easiest thing to do is
just have a flag day where blocks after a certain height are considered
to have a different epoch from the standard 1970 one when computing
their time. Boring, but it works fine and only needs to be updated every
few decades.


You're midstate idea is very clever though and could come in handy in
the future for different purposes. Eventually we should discuss this
with the ASIC manufacturers - if it can be implemented as a firmware or
FPGA upgrade in the field all the better.

-- 
'peter'[:-1]@petertodd.org
010a10e05e172442e0675a818d17b62c1ed041a4572002ca051e


signature.asc
Description: 

Re: [Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread John Dillon
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Thu, May 9, 2013 at 1:57 AM, Peter Todd p...@petertodd.org wrote:
 Remember that interpreting the timestamp on a block for the purposes of
 timestamping is a lot more subtle than it appears at first.

I actually just meant how Pieter Wuille was talking about a blocktime accurate
to only within 18 hours. :) But it is a nice writeup!

In any case, for many things simple relative ordering is enough rather than
absolute time.

 There are two types of timestamps possible: proofs that data existed
 before a time, and proofs that data existed after. With the former type
 the *later* the proof says the data existed, the more conservative the
 assumptions behind the proof. So simply adding two hours to the block's
 timestamp is relatively reasonable. (this assumes the attack managed to
 mine a single block, and all nodes have accurate clocks)

Nope. The attacker can make the timestamp on the block they mine as little as
the minimum from GetMedianTimePast(), and adding two hours to that number could
easily be well before true time.

What you probably need to do is some sort of median time calculation for the
blocks around your timestamp. The proof becomes probabalistic based on the % of
hashing power the attacker controls and in turn depends on if the time they
created their timestamp was of their own choosing.

IE, if you just want to create an inaccurate timestamp, but don't care when,
you can just mine blocks and wait until you get lucky. If you need to create an
inaccurate timestamp *now* the problem is much harder.

But all this analysis can be developed later, and data timestamped now. :)

 Back to the block header time... Frankly, the easiest thing to do is
 just have a flag day where blocks after a certain height are considered
 to have a different epoch from the standard 1970 one when computing
 their time. Boring, but it works fine and only needs to be updated every
 few decades.

Oh, right, yes, that is a much more simple idea and far less prone to bugs.

Many SPV clients wouldn't even need upgrades if they don't acturally validate
the blocks they receive and just look for the biggest PoW.


 You're midstate idea is very clever though and could come in handy in
 the future for different purposes. Eventually we should discuss this
 with the ASIC manufacturers - if it can be implemented as a firmware or
 FPGA upgrade in the field all the better.

Yes


Anyway, you are being distracted from what we were talking about before, get
back to work!
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBCAAGBQJRiwrGAAoJEEWCsU4mNhiPvYAH/3kjlg5diWeyYUJlKPKRpygQ
XAU8a2D3h9bABacmmhx5yW3AmtV0QqLgKlB76t41JB4O6Jer5FT8tBBwPnjDijtI
KrBWwPqNhVZiyTSDZQTF6BR1a0DDCZVtOXlpOTj6NL1+hy7NYTYsqxAVzS8QgZUH
RXt7QTYGrrmMbmm75NdWhK59mbv22UEaDHfDW0qqgSzdb7f1EQv1fou3MfKScQSd
7OsGU3k5PM+KQ/FGBy+r+07GY5yj85YMooGky0MCjtkOiU/qr+pxfs1uT2R8/512
TyZxzn1vtgWUEGOUeMCml+bgjUOvOgcIvAarzZmyLyAAY15S/LT8MvAr2RUjAfY=
=UDyE
-END PGP SIGNATURE-

--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] 32 vs 64-bit timestamp fields

2013-05-08 Thread Peter Todd
On Thu, May 09, 2013 at 02:33:11AM +, John Dillon wrote:
 On Thu, May 9, 2013 at 1:57 AM, Peter Todd p...@petertodd.org wrote:
  Remember that interpreting the timestamp on a block for the purposes of
  timestamping is a lot more subtle than it appears at first.
 
 I actually just meant how Pieter Wuille was talking about a blocktime accurate
 to only within 18 hours. :) But it is a nice writeup!
 
 In any case, for many things simple relative ordering is enough rather than
 absolute time.

Ah, shoot, I just realized we both got missed Pieter's point entirely:
he means to change the meaning of the header timestamp to be relative
time passed since the last block...

Well, it was a nice writeup! Thanks for the correction re:
probabalistic; you are absolutely correct.

-- 
'peter'[:-1]@petertodd.org
00fb6d0ed7479069edef10b8bc598783e9d94bdb5cf9ae6a5f1c


signature.asc
Description: Digital signature
--
Learn Graph Databases - Download FREE O'Reilly Book
Graph Databases is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development