I'm replying to https://github.com/bitcoinj/bitcoinj/issues/1259
> I present some ideas for action but more thoughts are needed. Testnet forks > can be very long, to the degree it will run beyond what can be handled with > SPVBlockStore, so we need to think about this. A solution for a very low spec > (memory/disk) wallet could maybe do a two-pass download from peers. (I am > unsure but does bitcoinj make use getheaders in normal catch up) > Peers do not recognize hashes in getblock-message Do you know *how* long exactly forks on testnet can be? Since mainnet forks are not that long (proof of works prevents them to be), I'd be ok with a trivial fix like simply increasing the size for the ring buffer (for testnet only!). > The hashes sent in the getblock message should help a peer to decide which > blocks the client needs to catch up with the chain. I noticed that Peers > response to getblocks was to send the 500 first blocks of the block chain. > These are not to much use of course. It will not change the state of of the > chain head in the wallet's store so the wallet will be stuck. > > Analysis: The bitcoinj seams to fill in 100 hashes starting from its chain > head in a linear fashion, and if all are on a fork that was discarded, the > Peer cannot find any common block except for the genesis block, thus it > starts there in the reply inv-message. Exactly; also see comment and implementation at org.bitcoinj.core.Peer.blockChainDownloadLocked(Sha256Hash). > Action: Use better set of hashes from the known blocks in the store (5000 for > SPVBlockStore). A better selection is proposed on the bitcoin wiki: "dense to > start, but then sparse". This helps but I ran into the next problem: > getdata 500 blocks do not trigger re-organize despite head on a dead fork Should be easy to do. The only problem is backwards scanning the block store is a bit expensive, but lets see. > Requesting blocks with better hashes can still leave the the wallet store > chain head unchanged. This will result in the same request for blocks again, > and the store head is effectively stuck. How did you test this? Did you already fix the above problem? > Analysis: This happens when the downloaded blocks, despite belong to the > correct chain, will not trigger a re-organize despite the head is a dead > chain. Why? It seems the special difficulty jumps on testnet can make a > branch of blocks have less total work despite being very much longer, e.g. > several 100 blocks longer than head. For some reason the network selected > this longer chain for many blocks. The reorg logic is tricky unfortunately and because reorgs are difficult to test I fear they are not tested well and they are prone to regressions. > Action: Not sure here: one way is to follow the current rules. In these cases > we need to download many blocks (1000s) to trigger a re-org. To make that > happen the store needs to track not only chain head but also what, at this > point, looks like a fork (less total work) to send a different getblocks and > getdata to get more blocks it has not downloaded. Note that getblocks can > easily be used to ask for multiple branches in one request. But, without > extending the SPVBlockStore this solution does not work as we run out of > space (currently to manage the reorg bitcoinj seems to need all blocks in > both branches back to the split point). > Another way is maybe to discard blocks back to the split point and try to > restart with getblocks to peers (but this would trust this peer more than > those that resulted in the current head). Does the transaction confidence > model allow this two step operation: first lowering head and total work, and > then following another branch that eventually will reach higher total work > much later? Currently I assume total work can only increase. I general, tx > confidence changes for these deep reorg events seems very hard to handle in > applications anyway. Any ideas? My gut feeling is that total work should only increase. I'd try increasing block store capacity first. -- You received this message because you are subscribed to the Google Groups "bitcoinj" group. To unsubscribe from this group and stop receiving emails from it, send an email to bitcoinj+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
