Re: [Bitcoin-development] Abusive and broken bitcoin seeders
I suspect it is something that is going to have to be dealt with in the future (I just don't know how yet). The web has managed to survive despite constant fast crawls being the norm for the past 10 years or so. I wouldn't worry too much about this unless you can prove that a big chunk of your nodes resources are going to answering ver queries. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Abusive and broken bitcoin seeders
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I may be able to provide some insight regarding request volume / abuse via my public node at http://statoshi.info My node receives a 'getaddr' request about every 50 seconds: http://i.imgur.com/XEpnWfG.png In terms of the 'addr' messages that it sends out, the volume is also low. This graph has 'inv' and 'tx' sent messages for comparison. http://i.imgur.com/keyitsS.png Now, these are just message volume and not actual resource usage, but I have a feeling that 'getaddr' requests are not resource intensive since it shouldn't be reading from disk. I could look into adding timing metrics around these requests if you think it could be useful. - - Jameson On 07/31/2014 06:37 AM, Mike Hearn wrote: I suspect it is something that is going to have to be dealt with in the future (I just don't know how yet). The web has managed to survive despite constant fast crawls being the norm for the past 10 years or so. I wouldn't worry too much about this unless you can prove that a big chunk of your nodes resources are going to answering ver queries. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJT2j2lAAoJEIch3FSFNiDcWqIH/i0W21cYFHyQZItSkyHezBER ExjudrLuXvTuRc/9b1OG7lJpK7IEYpCn0xXHGP3gv8gihq6lVEdZCFMXGWxU+eDv ECXppTTCUkofUjVInbU91eagXeRzK0UTbTrp2++hfLQIAv99B8mgSdoEcopP42Fd G197p/273lAPGVmNF31YPUcIbrhj0IzsiR1QaEEf1FEelaJ7MmU7YsUFUglajTqk 6+Uzcr6RcwLKAWVFAOA6VOeVwAMOQMwsniUAx6bYbqvgSHzRTllDDWW5rTaKh9+O rIhA3LvHpLh37xqTs6EvJb2Kn823e4Ax4Eoz3wqVvAyjNqWHRPjlXdXentHFN4Q= =R+Z1 -END PGP SIGNATURE- -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] [ANN] Armory 0.92 with Decentralized Multi-sig and Simulfunding
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/30/2014 09:50 PM, Alan Reiner wrote: (though, in the future, we hope to provide an optional service to help synchronize the data between parties) Before rolling your own service, it might be a good idea to add Bitmessage integration to provide the P2P communication layer. Even if you resolved to create such a service without creating any negative privacy or confidentially side effects, I'd be more inclined to trust Bitmessage to get that right in the long term, because the service you'd create isn't your primary product or core competency. - -- Support online privacy by using email encryption whenever possible. Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k -BEGIN PGP SIGNATURE- iQEcBAEBAgAGBQJT2ovhAAoJEMP3uyY4RQ21zw4H/3vjcZXP6e0/5IG745PDy/AC Br1ChlyQjBpU7X9CQfrxDmUUGs7HDrwLjd/SZAV1/PUUXXfE3nDr24hsF8+PlGex AiZhO7k92xfwRMWxMmcVVt/kuaOldHZUqHDUenT3drJ/bPnV+R3FJ9O6Ougu/YVy H2BRjpdPGrZx9NP/hE/7evA7rPF8pcshpMBiwq6RiHFdu/+2jcThFZoMIaJsAcif 1vZOzP6vTUKkr3E7tRt5ZQrdb4vvGxX+xMomm8fzPmV3GkpJ/Kyyypx+ovaH74V5 oXXg62XRz4lSziWV5Sp4p/18VjRyUkxwvfXXMt9sW6vNvRDxtJNP8/ZKpkMjO3s= =ClEd -END PGP SIGNATURE- 0x38450DB5.asc Description: application/pgp-keys -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire
I don't see how set reconciliation alone would be practical for condensed block exchange -- if the keys are txids it'd require a round trip to request the missing tx; if we could somehow get the What's the Difference approach to effectively operate on full transactions instead, the log(keysize) factor overhead would make any transactions not mutually-known very expensive to exchange (at keysize=32b, data would need to be 80% mutually-known just to break even). There's also the complication and/or overhead of establishing an expected block to reconcile with the actual block. The approach of remembering what invs have been transmitted both directions along each connection is less elegant; it requires remembering a lot of communication history, introducing a major point of statefulness to the protocol, and custom-compacting blocks for each peer. But it is also very effective at squeezing bytes, cheap in cpu cycles, and the implementation is fairly simple. The wealth of mutual knowledge already available in the current protocol allows accomplishing the goal of exchanging blocks efficiently by solving a much easier problem than its context-less cousin. I have my doubts that it is possible for even an optimal contextless solution to do as well as channel memory in terms of bytes exchanged or computational complexity -- you can't beat making use of the available information. I have an implementation of inv-history-tracking that uses a 2b/tx alternative to getdata for tx, and I've had that running between two nodes for ~2 weeks now. I've been working on a better implementation of that plus the sparseblock messages, and I'll have the sparseblock prototype (suitable for something like Gregory's remember-last-N approach) up and running in a couple of days or so. The prototype handles assigning compact identifiers to transactions and using those in block messages; there's a lot of bit-packing sort of tweaks that can be done that I'm not including in the initial prototype. The prototype will be able to log history-hit rates, so if we run a few sparseblocks nodes connected to each other for a while we should get a good idea of how much efficiency gain this provides, and how it can be improved. This approach even without the intensive bit packing has a total vtx transmission size of 2*nTxKnown + 1*nTxUnknown + nBytesTxUnknown, where only a small window of very recent transactions and any transactions that have fallen out of the history limit would be mutually known but not known to be known. It would be possible to nearly eliminate even that overhead for both known and unknown transactions with compact descriptions of block tx inclusion and ordering policies as Gavin brought up, for which something like scripts defining priority formulas would be a possible implementation (https://gist.github.com/kazcw/43c97d3924326beca87d#ordering-policy -- n.b. most of the rest of the gist is currently outdated). But since priority scripts are themselves more complicated than the rest of the sparseblock implementation, and basic sparseblocks achieve the vast majority of bandwidth savings, I think it's worth implementing sparseblocks without priority scripts now and then using priority scripts for sparseblocks2 along with all the other things they can do later. Set reconciliation does look like a great way to synchronize mempools. I've been thinking, contextless low-cost mempool exchange would enable a node to have one or more roaming peer slots -- connect to a node, fill in each other's mempools, move on to another peer. It seems like this would go a long way to mitigate potential pathological network topologies -- it would make it very difficult to sybil attack a node (barring an attacker in a position to spoof IP addresses), and if a serious bug or DoS attack caused the network to start to partition itself due to DoS bans, it only takes occasional roamers crossing the partition to keep both sides generally in sync. Efficient mempool synchronization would also increase the efficacy of channel-memory sparseblocks: it picks up transactions too old to have been exchanged via invs, and could also allow nodes to know exactly what transactions their peers have discarded. On Thu, Jul 31, 2014 at 8:31 AM, Gavin Andresen gavinandre...@gmail.com wrote: I've been reading up on set reconciliation algorithms, and thinking about constant-bandwidth propagation of new block announcements. Emin: the approach in this paper: What's the Difference? Efficient Set Reconciliation without Prior Context http://conferences.sigcomm.org/sigcomm/2011/papers/sigcomm/p218.pdf ... looks like it would be much better suited for Bitcoin's use case, because a) it looks much easier to implement (no polynomial math) b) CPU/latency versus bandwidth tradeoff looks better (somewhat higher bandwidth than Yaron's method, but much lower CPU/latency cost) Kaz: how much time do you have to work on this? Perhaps we can get a road-map for a prototype and
Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire
On Thu, Jul 31, 2014 at 1:47 PM, Kaz Wesley kezi...@gmail.com wrote: trip to request the missing tx; if we could somehow get the What's the Difference approach to effectively operate on full transactions instead I explain how to do this on the network block coding page. Given that you know the sizes and orders of the transactions (e.g. from a reconciliation step first), the sender sends non-syndromic forward error correcting code data somewhat larger than their estimate of how much data the user is missing. Then you drop the data you know into place and then recover the missing blocks using the fec. There is no overhead in this approach except for FEC blocks that are incompletely missing (and so must be completely discarded), and the need to have the transmitted the transaction list and sizes first. (note, that just more bandwidth, not an additional round trip). -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire
the need to have transmitted the transaction list [..] first 32 bits per transaction is at least double the communication overhead of the simple approach, and only offers a bound on the probability of needing a round trip. On Thu, Jul 31, 2014 at 2:29 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Thu, Jul 31, 2014 at 1:47 PM, Kaz Wesley kezi...@gmail.com wrote: trip to request the missing tx; if we could somehow get the What's the Difference approach to effectively operate on full transactions instead I explain how to do this on the network block coding page. Given that you know the sizes and orders of the transactions (e.g. from a reconciliation step first), the sender sends non-syndromic forward error correcting code data somewhat larger than their estimate of how much data the user is missing. Then you drop the data you know into place and then recover the missing blocks using the fec. There is no overhead in this approach except for FEC blocks that are incompletely missing (and so must be completely discarded), and the need to have the transmitted the transaction list and sizes first. (note, that just more bandwidth, not an additional round trip). -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire
On Thu, Jul 31, 2014 at 2:41 PM, Kaz Wesley kezi...@gmail.com wrote: the need to have transmitted the transaction list [..] first 32 bits per transaction is at least double the communication overhead of the simple approach, and only offers a bound on the probability of needing a round trip. (e.g. from a reconciliation step first) the list can be communicated in the space roughly equal to the size of the difference in sets plus coding the permutation from the permissible orderings. If you did have some simple approach that guaranteed that some transactions would be present, then you could code those with indexes... the FEC still lets you fill in the missing transactions without knowing in advance all that will be missing. (Also, the suggestion on the network block coding page of using part of a cryptographic permutation as the key means that for unknown transactions the transmission of the new unknown keys is always goodput— doesn't add overhead) It's only a bound but you can pick whatever bound you want, including— if you send data equal to the missing amount, then it'll be always successful, but no bandwidth savings. Though if the transport is unordered (e.g. UDP or non-blocking SCTP) even sending 100% of the missing amount can save time by eliminating a round trip that might otherwise be needed for a retransmission. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire
the FEC still lets you fill in the missing transactions without knowing in advance all that will be missing. I don't see why we need to solve that problem, since the protocol already involves exchanging the information necessary to determine (with some false positives) what a peer is missing, and needs to continue doing so regardless of how blocks are transmitted. Set reconciliation does have the benefit of eliminating a subset of those false positives and offering a finer-grained mechanism for defining what a node can choose to forget from its mempool than remember-last-N, but if we implement it for block transmission I don't see why we wouldn't also use it to synchronize mempool txes, and if mempools are synchronized we don't actually need to do it as part of block-transmission to get those benefits. As far as I can tell, channel memory sparseblocks achieve most of the possible bandwidth savings, and when FEC-based mempool synchronization is implemented its benefits can be applied to the sparseblocks by resetting the channel memory to the mutual mempool state each time mempool differences are exchanged. Am I missing a benefit to doing FEC at block forwarding time that can't be realized by FEC-based mempool synchronization, implemented separately from channel-memory based index-coding? On Thu, Jul 31, 2014 at 2:51 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Thu, Jul 31, 2014 at 2:41 PM, Kaz Wesley kezi...@gmail.com wrote: the need to have transmitted the transaction list [..] first 32 bits per transaction is at least double the communication overhead of the simple approach, and only offers a bound on the probability of needing a round trip. (e.g. from a reconciliation step first) the list can be communicated in the space roughly equal to the size of the difference in sets plus coding the permutation from the permissible orderings. If you did have some simple approach that guaranteed that some transactions would be present, then you could code those with indexes... the FEC still lets you fill in the missing transactions without knowing in advance all that will be missing. (Also, the suggestion on the network block coding page of using part of a cryptographic permutation as the key means that for unknown transactions the transmission of the new unknown keys is always goodput— doesn't add overhead) It's only a bound but you can pick whatever bound you want, including— if you send data equal to the missing amount, then it'll be always successful, but no bandwidth savings. Though if the transport is unordered (e.g. UDP or non-blocking SCTP) even sending 100% of the missing amount can save time by eliminating a round trip that might otherwise be needed for a retransmission. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire
On Thu, Jul 31, 2014 at 3:27 PM, Kaz Wesley kezi...@gmail.com wrote: the FEC still lets you fill in the missing transactions without knowing in advance all that will be missing. I don't see why we need to solve that problem, since the protocol already involves exchanging the information necessary to determine (with some false positives) what a peer is missing, and needs to continue doing so regardless of how blocks are transmitted. False positives, and if you have more than one peer— false negatives. (or a rule for what you must keep which is conservative in order to avoid creating huge storage requirements— but then also has false negatives). As far as I can tell, channel memory sparseblocks achieve most of the possible bandwidth savings, and when FEC-based mempool synchronization is implemented its benefits can be applied to the sparseblocks by resetting the channel memory to the mutual mempool state each time mempool differences are exchanged. Am I missing a benefit to doing FEC at block forwarding time that can't be realized by FEC-based mempool synchronization, implemented separately from channel-memory based index-coding? Yes, minimizing latency in the face of multiple peers. Otherwise no. And certantly no reason to to implement something simple first. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire
On Thu, Jul 31, 2014 at 4:18 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Thu, Jul 31, 2014 at 3:27 PM, Kaz Wesley kezi...@gmail.com wrote: the FEC still lets you fill in the missing transactions without knowing in advance all that will be missing. I don't see why we need to solve that problem, since the protocol already involves exchanging the information necessary to determine (with some false positives) what a peer is missing, and needs to continue doing so regardless of how blocks are transmitted. False positives, and if you have more than one peer— false negatives. (or a rule for what you must keep which is conservative in order to avoid creating huge storage requirements— but then also has false negatives). I think a rule for what to keep (along with the false-positive rate associated with the rule's conservativeness) is preferable to false negatives, since round-trip cost is potentially very high. The prototype I'm working on will be able to provide data on what the false-positive-missing-tx rate would look like with something like remember-last-N. There are various ways that rule could be upgraded to nearly eliminate the false-positive-missing rate, including learning what txes a peer has dropped via periodic mempool syncing, or specifying the rule explicitly with priority scripts, both of which have other benefits of their own. All of these changes synergize, but as long as the simplistic remembering rule yields worthwhile improvement over the current approach they can all be done independently as incremental improvements. I also really like the idea of the referring to transactions by ids that can themselves provide part of the tx data; I think that could also be added separately, on top of these other changes. (Gregory, your various wiki pages are basically my to-do list of things I'd like to work on.) The idea of mempool synchronization brings up the issue of transaction expiration, since lack of mempool syncing is currently the mechanism for tx expiry. I'm starting a discussion about how to address that in a separate thread; see deterministic transaction expiration. As far as I can tell, channel memory sparseblocks achieve most of the possible bandwidth savings, and when FEC-based mempool synchronization is implemented its benefits can be applied to the sparseblocks by resetting the channel memory to the mutual mempool state each time mempool differences are exchanged. Am I missing a benefit to doing FEC at block forwarding time that can't be realized by FEC-based mempool synchronization, implemented separately from channel-memory based index-coding? Yes, minimizing latency in the face of multiple peers. Otherwise no. And certantly no reason to to implement something simple first. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thu, Jul 31, 2014 at 05:58:23PM -0700, Kaz Wesley wrote: I have a proposal for a way to add finite and predictable lifespans to transactions in mempools: we d̶e̶s̶t̶r̶o̶y̶ ̶t̶h̶e̶ ̶r̶e̶s̶u̶r̶r̶e̶c̶t̶i̶o̶n̶ ̶h̶u̶b̶ use nLockTime and a new standardness rule. It could be done in stages, would not necessarily require even a soft fork, and does not cause problems with reorgs like the proposal in #3509: Anything that changes the semantics of nLockTime will do harm to existing and future applications that make use of nLockTime for things like refund transactions. In any case, why do transactions need finite lifespans in mempools? If you want to double-spend them with higher fees, then implement replace-by-fee. In any case, lifetimes will never be deterministic as not everyone runs the same software. Transactions would stop being relayed and drop out of mempools a fixed number of blocks from their creation; once that window had passed, the sender's wallet could begin to expect the transaction would not be confirmed. In case a reorg displaces a transaction until after its expiry height, a miner can still put it back in the blockchain; the expiry height is just a relay rule. Also, a user who needed to get their original expired transaction confirmed could still do so by submitting it directly to a miner with suitable policies. ...in which case someone will circumvent this IsStandard() rule by submitting expired transactions directly to miners with suitable policies. signature.asc Description: Digital signature -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thu, Jul 31, 2014 at 6:06 PM, Peter Todd p...@petertodd.org wrote: Anything that changes the semantics of nLockTime will do harm to existing and future applications that make use of nLockTime for things like refund transactions. I think this would be compatible with most uses of nLockTime -- e.g., at the time a refund transaction becomes broadcastable, its beneficiary would usually have no reason to wait for a long time before broadcasting it; if they did so (probably because they weren't online to redeem the refund), they'd need to use the submit-directly-to-miner option, but they wouldn't lose their refund. In any case, why do transactions need finite lifespans in mempools? If you want to double-spend them with higher fees, then implement replace-by-fee. Perpetuating transactions that have been in mempools for a long time and are not being confirmed has been cited as a reason for nodes not to exchange mempools (#3721, #1833, #3722); it's been implied that it would be desirable for wallets to wait until a transaction had had a chance to be accepted before double-spending with a higher fee (#3722); and an unconfirmed transaction-age-based policy for preventing mempool accumulation has been advocated (#3753, #3722) [I hope my summarization is not misrepresenting anyone's opinions here; please see the arguments made in the actual comments on the bugs]. These discussions are mostly fairly old, but I don't know of any changes that have been made that provide alternative answers to these concerns mentioned by at least three different developers. In any case, lifetimes will never be deterministic as not everyone runs the same software. That's true, but none of the benefits of these changes require the policy to be unanimous; most of the effect could be provided by most of the network following these rules. Transactions would stop being relayed and drop out of mempools a fixed number of blocks from their creation; once that window had passed, the sender's wallet could begin to expect the transaction would not be confirmed. In case a reorg displaces a transaction until after its expiry height, a miner can still put it back in the blockchain; the expiry height is just a relay rule. Also, a user who needed to get their original expired transaction confirmed could still do so by submitting it directly to a miner with suitable policies. ...in which case someone will circumvent this IsStandard() rule by submitting expired transactions directly to miners with suitable policies. Yes, that is a feature. None of the benefits of transaction expiration rely on expiration being final, and any possible downsides of expiration are largely mitigated by the option still being available to get expired transactions mined. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
It would make more sense to introduce a new script opcode that pushes the current block height onto the operand stack. Then you could implement arbitrary logic about which blocks the transaction can be valid in. This would require that the client revalidate all transactions in its mempool (really, only those making use of this opcode) whenever the chain tip changes. On Thursday, 31 July 2014, at 5:58 pm, Kaz Wesley wrote: There is currently little in place for managing transaction lifetime in the network's mempools (see discussion in github in #3722 mempool transaction expiration, and it seems to be a major factor blocking some mempool exchange, see #1833/1918, #3721). Expiry per-node a certain amount of wall time after receipt has been proposed, but that's a fragile mechanism -- a single node could keep all relayable transactions alive forever by remembering transactions until most nodes have dropped them and then releasing them back into the wild. I have a proposal for a way to add finite and predictable lifespans to transactions in mempools: we d̶e̶s̶t̶r̶o̶y̶ ̶t̶h̶e̶ ̶r̶e̶s̶u̶r̶r̶e̶c̶t̶i̶o̶n̶ ̶h̶u̶b̶ use nLockTime and a new standardness rule. It could be done in stages, would not necessarily require even a soft fork, and does not cause problems with reorgs like the proposal in #3509: 1. start setting nLockTime to the current height by default in newly created transactions (or slightly below the current height, for reorg-friendliness) 2. once users have had some time to upgrade to clients that set nLockTime, start discouraging transactions without nLockTime -- possibly with a slightly higher fee required for relay 3. start rate-limiting relay of transactions without an nLockTime (maybe this alone could be used to achieve [2]) 4. add a new IsStandard rule rejecting transactions with an nLockTime more than N blocks behind the current tip (for some fixed value N, to be determined) Transactions would stop being relayed and drop out of mempools a fixed number of blocks from their creation; once that window had passed, the sender's wallet could begin to expect the transaction would not be confirmed. In case a reorg displaces a transaction until after its expiry height, a miner can still put it back in the blockchain; the expiry height is just a relay rule. Also, a user who needed to get their original expired transaction confirmed could still do so by submitting it directly to a miner with suitable policies. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thu, Jul 31, 2014 at 6:38 PM, Matt Whitlock b...@mattwhitlock.name wrote: It would make more sense to introduce a new script opcode that pushes the current block height onto the operand stack. Then you could implement arbitrary logic about which blocks the transaction can be valid in. This would require that the client revalidate all transactions in its mempool (really, only those making use of this opcode) whenever the chain tip changes. Transactions that become invalid later are have pretty severe consequences because they might mean that completely in an absence of fraud transactions are forever precluded due to a otherwise harmless reorg. While there may be uses for that, the resulting outputs should be considered differently fungible— like coinbases which are immature— and should probably be only used with great caution... not as a mechanism for ordinary transactions. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thursday, 31 July 2014, at 7:28 pm, Gregory Maxwell wrote: On Thu, Jul 31, 2014 at 6:38 PM, Matt Whitlock b...@mattwhitlock.name wrote: It would make more sense to introduce a new script opcode that pushes the current block height onto the operand stack. Then you could implement arbitrary logic about which blocks the transaction can be valid in. This would require that the client revalidate all transactions in its mempool (really, only those making use of this opcode) whenever the chain tip changes. Transactions that become invalid later are have pretty severe consequences because they might mean that completely in an absence of fraud transactions are forever precluded due to a otherwise harmless reorg. I understand what you're saying, but I don't understand why it's a problem. Transactions shouldn't be considered final until a reasonable number of confirmations anyway, so the possibility that an accepted transaction could become invalid due to a chain reorganization is not a new danger. Ordinary transactions can similarly become invalid due to chain reorganizations, due to inputs already having been spent in the new branch. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thu, Jul 31, 2014 at 8:26 PM, Matt Whitlock b...@mattwhitlock.name wrote: I understand what you're saying, but I don't understand why it's a problem. Transactions shouldn't be considered final until a reasonable number of confirmations anyway, so the possibility that an accepted transaction could become invalid due to a chain reorganization is not a new danger. Ordinary transactions can similarly become invalid due to chain reorganizations, due to inputs already having been spent in the new branch. A distinction there is that they can only become invalid via a conflict— replaced by another transaction authored by the prior signers. If no other transaction could be created (e.g. you're a multisigner and won't sign it again) then there is no such risk. It now introduces chance events (act of god) into the mix where they they didn't exist before. Basically it takes was what is a very loose one way coupling and makes it much tighter. I'm sure if you spend a bit thinking you can come up with some more corner cases that it would expose— e.g. flooding the network with unrelated high fee transactions in order to push a transaction out to where it can never be mined at all. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
