Re: [Bitcoin-development] Economics of information propagation

2014-04-21 Thread Paul Lyon
I haven't done the math on this, so it may be a terrible idea. :)
I've been wondering if block propagation times could also be improved by 
allowing peers to request the list of transaction hashes that make up a block, 
and then making a follow-up request to only download any transactions not 
currently known. I'm not sure what percentage of transactions a node will 
usually already have when it receives a new block, but if it's high I figure 
this could be beneficial.

 Date: Mon, 21 Apr 2014 09:00:09 -0700
 From: m...@monetize.io
 To: p...@petertodd.org; jonathan.le...@sant.ox.ac.uk
 CC: bitcoin-development@lists.sourceforge.net
 Subject: Re: [Bitcoin-development] Economics of information propagation
 
 That wasn't what I was saying. Right now the primacy of a block is
 determined by the time at which the `block` message is received, which
 is delays due to both the time it takes to transmit the block data and
 the time it takes to validate. Headers-first, on the other hand, has the
 option of basing primacy on the time the block header is received, which
 is O(1) time to transmit and to SPV-validate. Mining on that block
 doesn't actually commence until the full block is received and validated.
 
 To see how this works, take an example: two blocks with a common parent
 are found relatively close to each other, block A and block B. A is
 found first but is a large block with the maximum block size and many
 slow scripts. B is found a few seconds later and is an empty block. In
 the current regime it is entirely possible that block B, the later but
 smaller block, would get received and processed first by more mining
 peers than the larger block A, exactly as described in Jonathan Levin's
 email.
 
 With headers-first, however, the cost of propagation of the block header
 is the same and we should expect block A to win out over block B nearly
 every time. Miners will continue working on the old, known valid parent
 block until the contents of block A are received and processed. So the
 smaller block B is still found, and since it's data moves across the
 network faster, miners even briefly mine on block B. But as soon as they
 receive and process the contents of block A, they switch to that.
 
 The earlier, larger block A will only become stale if *two* blocks are
 found in the extra time it takes for block A to propagate the network.
 That is a substantially different risk, and probably a negligible
 concern to most miners.
 
 On 04/20/2014 09:06 PM, Peter Todd wrote:
  That is mistaken: you can't mine on top of just a block header,
  leaving small miners disadvantaged as they are earning no profit
  while they wait for the information to validate the block and update
  their UTXO sets. This results in the same problem as before, as the
  large pools who mine most blocks can validate either instantly - the
  self-mine case - or more quickly than the smaller miners.
  
  Of course, in reality smaller miners can just mine on top of block
  headers and include no transactions and do no validation, but that is
  extremely harmful to the security of Bitcoin.
 
 --
 Start Your Social Network Today - Download eXo Platform
 Build your Enterprise Intranet with eXo Platform Software
 Java Based Open Source Intranet - Social, Extensible, Cloud Ready
 Get Started Now And Turn Your Intranet Into A Collaboration Platform
 http://p.sf.net/sfu/ExoPlatform
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development
  --
Start Your Social Network Today - Download eXo Platform
Build your Enterprise Intranet with eXo Platform Software
Java Based Open Source Intranet - Social, Extensible, Cloud Ready
Get Started Now And Turn Your Intranet Into A Collaboration Platform
http://p.sf.net/sfu/ExoPlatform___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Why are we bleeding nodes?

2014-04-07 Thread Paul Lyon
I hope I'm not thread-jacking here, apologies if so, but that's the approach 
I've taken with the node I'm working on.
Headers can be downloaded and stored in any order, it'll make sense of what the 
winning chain is. Blocks don't need to be downloaded in any particular order 
and they don't need to be saved to disk, the UTXO is fully self-contained. That 
way the concern of storing blocks for seeding (or not) is wholly separated from 
syncing the UTXO. This allows me to do the initial blockchain sync in ~6 hours 
when I use my SSD. I only need enough disk space to store the UTXO, and then 
whatever amount of block data the user would want to store for the health of 
the network.
This project is a bitcoin learning exercise for me, so I can only hope I don't 
have any critical design flaws in there. :)

From: ta...@bitsofproof.com
Date: Mon, 7 Apr 2014 21:20:31 +0200
To: gmaxw...@gmail.com
CC: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Why are we bleeding nodes?


Once headers are loaded first there is no reason for sequential loading. 
Validation has to be sequantial, but that step can be deferred until the blocks 
before a point are loaded and continous.
Tamas Blummerhttp://bitsofproof.com


On 07.04.2014, at 21:03, Gregory Maxwell gmaxw...@gmail.com wrote:On Mon, Apr 
7, 2014 at 12:00 PM, Tamas Blummer ta...@bitsofproof.com wrote:
therefore I guess it is more handy to return some bitmap of pruned/full
blocks than ranges.

A bitmap also means high overhead and— if it's used to advertise
non-contiguous blocks— poor locality, since blocks are fetched
sequentially.



--
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
  --
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test  Deployment 
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Missing fRelayTxes in version message

2013-06-19 Thread Paul Lyon
I’m also running into this exact same issue with my parser, now I understand 
why the relay field behavior I was seeing doesn’tmatch the wiki.


So to parse a version message, you can’t rely on the protocol version? You have 
to know how long the payload is, and then parse the message accordingly? I 
agree with Turkey Breast, this seems a bit sloppy to me.


Paul


P.S. I’ve never used a dev mailing list before and I want to get involved with 
the Bitcoin dev community, so let me know if I’m horribly violating any mailing 
list etiquette. 



From: Mike Hearn
Sent: ‎Wednesday‎, ‎June‎ ‎19‎, ‎2013 ‎7‎:‎43‎ ‎AM
To: Turkey Breast
Cc: bitcoin-development@lists.sourceforge.net


Bitcoin-Qt on master does send it now although it doesn't affect anything, but 
as old pre-filtering versions will continue to exist, you'll always have to be 
able to deserialize version messages without it.



Bitcoin version messages have always had variable length, look at how the code 
is written in main.cpp. If you didn't experience issues until now all it means 
is that no sufficiently old nodes were talking to yours.




The standard does not say it should appear. Read it again - BIP 37 says about 
the new version message field:


If false then broadcast transactions will not be announced until a 
filter{load,add,clear} command is received. If missing or true, no change in 
protocol behaviour occurs.





On Wed, Jun 19, 2013 at 12:33 PM, Turkey Breast turkeybre...@yahoo.com wrote:




It's a problem if you work with iterators to deserialize the byte stream. Even 
failing that, it's just sloppy programming. What happens in the future when new 
fields are added to the version message? It's not a big deal to say that this 
protocol version has X number of fields, that (higher) protocol version message 
has X + N number of fields. Deterministic number of fields per protocol version 
is sensical and how Bitcoin has been for a long time.




And yes, it was a problem for me that caused a lot of confusion why this byte 
didn't exist in many version messages despite the standard saying it should and 
the code in bitcoind indicating it should. Nowhere was this written. It doesn't 
help other implementations to have an unclear behaviour that depends on some 
magic from one implementation.










From: Mike Hearn m...@plan99.net
To: Turkey Breast turkeybre...@yahoo.com 

Cc: bitcoin-development@lists.sourceforge.net 
bitcoin-development@lists.sourceforge.net 
Sent: Wednesday, June 19, 2013 11:39 AM


Subject: Re: [Bitcoin-development] Missing fRelayTxes in version message

 





It has to be optional because old clients don't send it, obviously.



Why is this even an issue? There's no problem with variable length messages in 
any codebase that I'm aware of. Is this solving some actual problem?




On Wed, Jun 19, 2013 at 12:30 AM, Turkey Breast turkeybre...@yahoo.com wrote:




That's me. I never said to make all messages fixed length. I said to make a 
fixed number of fields per protocol. So given a protocol version number, you 
know the number of fields in a message. This is not only easier for parsing 
messages, but just good practice. I don't see why a 1 byte flag needs to be 
optional anyway.













From: Mike Hearn m...@plan99.net
To: Turkey Breast turkeybre...@yahoo.com 
Cc: Bitcoin Dev bitcoin-development@lists.sourceforge.net 
Sent: Tuesday, June 18, 2013 9:48 PM
Subject: Re: [Bitcoin-development] Missing fRelayTxes in version message
 






It's not a bug (although there was recently a change to make bitcoind/qt always 
send this field anyway). 



I don't know where Amir is going with BIP 60. Version messages have always been 
variable length. There's nothing inherent in the Bitcoin protocol that says all 
messages are fixed length, indeed, tx messages are allowed to have arbitrary 
data appended after them that gets relayed.




On Tue, Jun 18, 2013 at 7:45 PM, Turkey Breast turkeybre...@yahoo.com wrote:




See this BIP. I'm not sure if this is a bug or what, but it would be good if 
messages always had a fixed number of fields per protocol version.




https://en.bitcoin.it/wiki/BIP_0060#Code_Updates




This BIP details everything that needs to be done and proposes a protocol 
upgrade.


--
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development





--
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net