Re: [Bitcoin-development] Decentralizing mining
On Fri, Jun 14, 2013 at 4:06 PM, Peter Todd p...@petertodd.org wrote: It strikes me that this would work best if the pool has a mempool with child-pays-for-parent support where conflicts *are* allowed. IE you record whatever transactions you know about, conflicting or not, calculate which ones gives you the most fees/kb, and then figure out which set of non-conflicting ones are optimal. Of course, optimal is the knapsack problem... Now you can easilly tell the miners working on shares for you which tx's would be optimal if they wish to know, and at the same time whatever shares they send you are most likely to include transactions in your mempool inventory, and thus they can send just tx hashes to reduce bandwidth. Part of the broader issue that when we send peers INV advertisements we should be telling them what the fee/kb is so our peers can prioritize properly. That'd also help for the case where you want to broadcast two transactions in a row, but the pair is only profitable because the second is paying the fee for the first. Interesting proposals, particularly this last. The net result impact is, however, that which was criticized in at least one forum thread: replace-with-higher-fee. Speaking of, the way we tell peers about new blocks is really suboptimal: we tell every peer, in no particular order, about a new block via a block INV message, and then we give them the new block in parallel. I was looking through comp-sci papers on optimal flood-fill/gossip algorithms for random graph networks and it appears that optimal is to spend all your bandwidth to send the message to your fastest peer first, followed by your next fastest and so on. This works best because you get the exponential growth scaling faster by propagating the message as deep as possible in the network, and it then can flood outwards from there. Just sorting the peer list by #inv-recevied/time when doing INV pushes and when attending to incoming messages would probably be a big improvement. In terms of packet size, I would like to look into the network-wide costs of simply broadcasting block header + coinbase TX + TX list. I bet miners would love to opt into that. If the share does meet difficulty, hand it to both the pool and the local bitcoind. Should hand it to the pool first though, because the pool likely has the fastest block propagation, then hand it to local bitcoind. An optimized version may want to have some record of measured bandwidth - this applies Bitcoin in general too, although also has other issues. Currently, BFGMiner is doing submission to the pool, waiting for a response, then submitting to a local bitcoind. This is because the pool might need to receive/record the share before it processes the block on bitcoind, or you could lose credit for it. The response from the pool is rather small (a single TCP packet), so this shouldn't delay much longer. Right, I guess the pool wants to be sure you were actually the one who found the share, rather than just someone who was lucky enough to see it on the network and submitted it as your own. ## Reducing bandwidth How about for normal shares we just pass the block header, and have the pool randomly pick a subset of transactions to audit? Any fraud cancels the users shares. This will work best in conjunction with a UTXO proof tree to prove fees, or by just picking whole shares randomly to audit. Might as well just use higher difficulty shares (each one audited) for the same effect. Block proposals allow the miner to tell the pool its transaction set once (per txset change) for any number of shares. That's a good point - the current practice most pools seem to follow of about a share per second seems very excessive to me. On the other hand, it does have good user optics. The right solution might be something akin to P2Pool where the UI level is telling the user shares are being found so it's clear stuff is happening, but under the hood only a small subset are ever sent to the pool. With the onslaught of ASIC mining, most big pools are past a share per second. Variable difficulty or set-to-higher-difficulty quickly became the norm, almost out of necessity. Personally, I think most pools should target at _least_ 5-10 seconds per share, no matter the strength of the miner. -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Decentralizing mining
On Mon, Jun 17, 2013 at 11:16:01AM -0400, Jeff Garzik wrote: Part of the broader issue that when we send peers INV advertisements we should be telling them what the fee/kb is so our peers can prioritize properly. That'd also help for the case where you want to broadcast two transactions in a row, but the pair is only profitable because the second is paying the fee for the first. Interesting proposals, particularly this last. The net result impact is, however, that which was criticized in at least one forum thread: replace-with-higher-fee. Actually the two are orthogonal: a low-priority no-fee tx might result because it was from a customer paying a merchant via the payment protocol. The merchant can then respend that tx with a fee to cover both, but with the current mempool arrangement if the no-fee tx load is high actually getting that first tx to propagate so the second can will be difficult. A nice way to do this would be to accept tx's into your mempool indiscriminately but delay broadcasting INV messages until you find child tx's that make the low-profit ones worth mining. When you do find a child with a sufficiently high fee, send an INVGROUP message to notify your peers of the new opportunity. Different nodes will have different ideas of what priority TX deserves to be broadcast, but here provided the group meets the threshold a peer will always find out. Speaking of, the way we tell peers about new blocks is really suboptimal: we tell every peer, in no particular order, about a new block via a block INV message, and then we give them the new block in parallel. I was looking through comp-sci papers on optimal flood-fill/gossip algorithms for random graph networks and it appears that optimal is to spend all your bandwidth to send the message to your fastest peer first, followed by your next fastest and so on. This works best because you get the exponential growth scaling faster by propagating the message as deep as possible in the network, and it then can flood outwards from there. Just sorting the peer list by #inv-recevied/time when doing INV pushes and when attending to incoming messages would probably be a big improvement. In terms of packet size, I would like to look into the network-wide costs of simply broadcasting block header + coinbase TX + TX list. I bet miners would love to opt into that. Whether or not that is a improvement is a really complex question, even without taking failure into account. If you agressively prioritize peers that are the most connected and keep your # of peers reasonably low you can afford the memory to keep track of what tx's your peers already know about so to save on round trips for TX hash's they don't have. On the other hand if you have a large number of peers and can't do that, or need to cut down on bandwidth used up by the INV floods and have a probabalistic scheme, you are risking more round-trip latency. Not to mention the nasty problem of how *relying* on TX hashes to keep your bandwidth down means that anything disrupting that system suddenly has a big impact on the network. I don't think we really understand all the nuances of that - look at how few people realize that you need multiples of average bandwidth to have sufficient emergency bandwidth available to catch up in the event of a chain fork. -- 'peter'[:-1]@petertodd.org 00a1c290ce20953d864a4b9c603abc8a9c77a04429c89c5e9fac signature.asc Description: Digital signature -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] bitcoinj 0.9
I'm pleased to announce the release of bitcoinj 0.9, a Java library for working with the Bitcoin protocol. Both simplified and full verification are supported. BitcoinJ has been used to create everything from end-user wallet apps to network crawlers to SatoshiDice. To get bitcoinj 0.9, check out our source from git and then run *git fetch --all; git checkout **67b187c4c4c4*. This will place you on the 0.9 branch in a secure manner. The roots of trust are the announcement sent to bitcoinj-announce (which is signed by the google.comDKIM key) and the Maven page of the bitcoinj website. This paragraph is signed with the same key as the previous releases (16vSNFP5Acsa6RBbjEA7QYCCRDRGXRFH4m). In addition, the 0.9 release is signed by Andreas Schildbach (GPG key id 0x8B877A60, accessible via http://bitcoin.org/andreas_schildbach.asc) and can be verified with *git tag -v 0.9* once you have his key. Signature for the last paragraph: IEVMFkGVfE5Q7mezpNc2srdMXMkE66AEW2g7AtWa2KGa2PcK5ehqGbKPOWaL2oftcN/939VHWViMLnCKGrS3E9g= We have a new article in the documentation library, Working with contractshttps://code.google.com/p/bitcoinj/wiki/WorkingWithContracts. It shows how to create and use multi-signature transactions, signed by different parties, using a simple API. *New in this release* - Thanks to Matt Corallo, we now have a basic *fee solver* that will attach the correct (minimum) fee per kilobyte to a created transaction using the 0.8.2+ fee rules. Note that there's no attempt to minimize the size of a calculated transaction and thus fee, but some other optimisations are applied. By default bitcoinj will always attach a fee, to learn how to customise this refer to the article *Working with the wallet*. - The wallet's re-org handling code was rewritten and simplified. - A new class, WalletAppKit, simplifies the process of instantiating all the objects and files that are needed to run a basic app that can send/receive money. - Add optional support for Pieter Wiulle's native secp256k1 implementation, which is significantly faster than Bouncy Castle. - Improvements to coin selection in the wallet. - Many new functions and minor API improvements, for instance, it's now easier to tell the wallet to allow spending of unconfirmed coins. - A new ScriptBuilder class simplifies the process of constructing various kinds of scripts. - A new block importer tool can parse bitcoind block files and process them, which is faster than streaming them over a network connection. - Support for the regtest mode added by the C++ side pull req 2632. This makes app development and testing easier by eliminating the need to wait for a block. - Many bug fixes and testing improvements. *API changes* - NetworkParameters has now been refactored into separate classes. - Wallet extensions have been tweaked, please refer to the javadocs for details. - Many other minor additions and changes that are mostly backwards compatible. *Known issues* * * Please see the limitations and missing features pagehttps://code.google.com/p/bitcoinj/wiki/Limitations on our website. -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Optional wallet-linkable address format - Payment Protocol
_*Goal*_: An alternative address format made possible by BIP 32, which
allows one to specify a Wallet ID and One-time payment code, instead
of the standard one-use Base58-Hash160 addresses. This allows parties
with a persistent relationship to be able to prove that payment
addresses they provide each other are linked to a particular wallet,
reducing exposure to MitM attacks without the need for SSL or a web of
trust, and without compromising the privacy of either party.For
instance, this could be used between businesses that frequently do
business, by exchanging and verifying public keys beforehand, or could
be used by an exchange to identify if a customer withdrawal address is
related to their last deposit address, and if not enforce extra
authentication measures.
_*Background*__:_
I haven't been following the payment protocol discussions/development
much, so I apologize if this has already been addressed. I'm calling
it wallet-linkable addresses, which would be an optional second form
for sending someone your address. With BIP 32, the address is computed
by the payee (the person sending the address to receive money):
Standard Address ~ Base58(0x00 || hash160(PubKeyParent *
Multiplier[i]) || checksum)
What I'd like to do is have the option, when specifying an address
through the payment protocol, to send *just* the {PublicKeyParent,
Multiplier[i]} and let the receiver of that address compute the address
on their own. This is no significant burden on the receiver, but it
does provide the useful property that they can recognize when addresses
specified in this way come from the same wallet -- because the
PubKeyParent will be the same. Remember, this is _optional_ for the
person providing the address.
One nice, accidental feature of BIP 32 is that the Multiplier[i] used
above does not actually reveal the chaincode (I think Pieter started
calling it the tweak). It is derived from the chaincode but doesn't
reveal it. Therefore, the payer sees the parent public key, but that's
not useful to derive any of the other addresses unless they also have
the chaincode. But they can verify that the PublicKeyParent is
identical between transactions, and thus is accessible only to that
wallet. It allows them validate a specific address provided by the
payee, but not generate or identify any other addresses.
*_Use Cases:_*
(1) So, just like with PGP/GPG, when two parties decide they will start
a relationship, they can start by exchanging the public keys of their
wallet and verify them in a reliable manner. After that, when one party
requests a payment address from the other, they can optionally send
{PubKey, Multiplier}, and the payer's software will identify the owner
of that address, or let you select who you think the address belongs to
and it will verify it. If the payee's system is compromised and address
is replaced, the address received by the payer won't validate. This
doesn't help if the side sending the money is compromised.
(2) When a customer first provides a deposit to an exchange, it will
send money from an address in their wallet and the software will provide
the exchange the {PubKey,Mult}. When the customer later provides a
withdrawal address, the site can automatically trust the address as long
it is provided in the alternate form and the public keys match. If they
don't, it might be the same customer just requesting a withdrawal to a
different wallet, which is fine, but they'll have to go through an extra
verification step to do so.
_*Downsides:*_
Multi-sig/P2SH - The only way this works with P2SH, violates one of the
goals of P2SH slightly, but may not matter much if it's all done under
the hood by the software. Instead of providing a 20-byte hash of a
script, you provide all the public keys and multipliers for the
individual addresses. The payer's software automatically verifies all
addresses and creates the P2SH script itself (after a divine decree that
public keys will always be sorted lexicographically in the multi-sig
script). The blockchain still benefits from the compression of moving
the bulky scripts to the TxIn, but it does require revealing more
information than is necessary for the payer to pay the payee. But it
may not /really/ be a problem, given the benefits. It might just be
slightly longer strings to exchange during initialization and for each
transaction.
I have various reasons I'd like to use this, and it'd be nice to have
some community backing, so I don't have to twist anyone's arm to trust
me that it's legit.
-Alan
--
This SF.net email is sponsored by Windows:
Build for Windows Store.
http://p.sf.net/sfu/windows-dev2dev___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
