Re: [Bitcoin-development] deterministic transaction expiration
Having explored more drastic approaches, it looks like Kaz' basic idea stands well. His #1... 1. start setting nLockTime to the current height by default in newly created transactions (or slightly below the current height, for reorg-friendliness) is already implemented in bitcoin-qt #2340, and a final call on merging it was already sent to this list. After some thought I agree with its policy of eventually setting nLockTime at current-height + 1 by default. This is the best reasonably expected height of any tx created right now. It discourages fee-sniping, and if a reorg happens anyway, it won't actually delay inclusion of tx beyond the reasonable expectation sans reorg. However right now, #2340 takes a very cautious approach and sets to current-height - 10 by default, with randomness to mitigate worries about loss of privacy. Kaz' #2, #3 and #4 are future actions. #4 only goes most of the way ... 4. add a new IsStandard rule rejecting transactions with an nLockTime more than N blocks behind the current tip (for some fixed value N, to be determined) ... a janitor mechanism is desirable to purge mempool of txes more than N behind current-height. Nodes dropping a tx N blocks after they became eligible to be mined (the meaning of nLockTime) makes sense. It is not an overloading or new use for nLockTime, but a logical extension of it. As Kaz pointed out, this solves a big problem with expiring by locally measured age: unintentional resurrection. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Fri, Aug 8, 2014 at 1:38 PM, Tom Harding t...@thinlink.com wrote: 4. add a new IsStandard rule rejecting transactions with an nLockTime more than N blocks behind the current tip (for some fixed value N, to be determined) It cannot be assumed that transaction creation time and transaction publish-to-outside-world time are the same, even though they often are. -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
A new network tx field would have the same problem, right? With a child-refreshes-parent policy, someone wishing to redeem a transaction that has passed its relay window without being confirmed could still do so. On Aug 8, 2014 11:16 AM, Jeff Garzik jgar...@bitpay.com wrote: On Fri, Aug 8, 2014 at 1:38 PM, Tom Harding t...@thinlink.com wrote: 4. add a new IsStandard rule rejecting transactions with an nLockTime more than N blocks behind the current tip (for some fixed value N, to be determined) It cannot be assumed that transaction creation time and transaction publish-to-outside-world time are the same, even though they often are. -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
How is eventual expiration of a tx that started life with an nLockTime in the future breaking, any more than any other tx expiring? On 8/6/2014 6:54 AM, Mike Hearn wrote: We could however introduce a new field in a new tx version. We know we need to rev the format at some point anyway. On Wed, Aug 6, 2014 at 2:55 PM, Jeff Garzik jgar...@bitpay.com mailto:jgar...@bitpay.com wrote: ...and existing users and uses of nLockTime suddenly become worthless, breaking payment channel refunds and other active uses of nLockTime. You cannot assume the user is around to rewrite their nLockTime, if it fails to be confirmed before some arbitrary deadline being set. On Wed, Aug 6, 2014 at 12:01 AM, Tom Harding t...@thinlink.com mailto:t...@thinlink.com wrote: ... If nLockTime is used for expiration, transaction creator can't lie to help tx live longer without pushing initial confirmation eligibility into the future. Very pretty. It would also enable fill or kill transactions with a backdated nLockTime, which must be confirmed in a few blocks, or start vanishing from mempools. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
...because nLockTime is the exact opposite of expiration. A locked TX begins life invalid, and becomes valid (not-expired) after nLockTime passes. A new field containing expiration time would work. On Wed, Aug 6, 2014 at 10:44 AM, Tom Harding t...@thinlink.com wrote: How is eventual expiration of a tx that started life with an nLockTime in the future breaking, any more than any other tx expiring? On 8/6/2014 6:54 AM, Mike Hearn wrote: We could however introduce a new field in a new tx version. We know we need to rev the format at some point anyway. On Wed, Aug 6, 2014 at 2:55 PM, Jeff Garzik jgar...@bitpay.com wrote: ...and existing users and uses of nLockTime suddenly become worthless, breaking payment channel refunds and other active uses of nLockTime. You cannot assume the user is around to rewrite their nLockTime, if it fails to be confirmed before some arbitrary deadline being set. On Wed, Aug 6, 2014 at 12:01 AM, Tom Harding t...@thinlink.com wrote: ... If nLockTime is used for expiration, transaction creator can't lie to help tx live longer without pushing initial confirmation eligibility into the future. Very pretty. It would also enable fill or kill transactions with a backdated nLockTime, which must be confirmed in a few blocks, or start vanishing from mempools. -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
+1 for the new field, overloading fields with new meaning is definitely not a good idea. Something like nExpireAt with a block height sounds reasonable to me, but we need to document that the usual caveats with blockchain reorgs apply. On Wed, Aug 6, 2014 at 4:08 PM, Jeff Garzik jgar...@bitpay.com wrote: ...because nLockTime is the exact opposite of expiration. A locked TX begins life invalid, and becomes valid (not-expired) after nLockTime passes. A new field containing expiration time would work. On Wed, Aug 6, 2014 at 10:44 AM, Tom Harding t...@thinlink.com wrote: How is eventual expiration of a tx that started life with an nLockTime in the future breaking, any more than any other tx expiring? On 8/6/2014 6:54 AM, Mike Hearn wrote: We could however introduce a new field in a new tx version. We know we need to rev the format at some point anyway. On Wed, Aug 6, 2014 at 2:55 PM, Jeff Garzik jgar...@bitpay.com wrote: ...and existing users and uses of nLockTime suddenly become worthless, breaking payment channel refunds and other active uses of nLockTime. You cannot assume the user is around to rewrite their nLockTime, if it fails to be confirmed before some arbitrary deadline being set. On Wed, Aug 6, 2014 at 12:01 AM, Tom Harding t...@thinlink.com wrote: ... If nLockTime is used for expiration, transaction creator can't lie to help tx live longer without pushing initial confirmation eligibility into the future. Very pretty. It would also enable fill or kill transactions with a backdated nLockTime, which must be confirmed in a few blocks, or start vanishing from mempools. -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 6 August 2014 08:17:02 GMT-07:00, Christian Decker decker.christ...@gmail.com wrote: +1 for the new field, overloading fields with new meaning is definitely not a good idea. To add a new field the best way to do it is create a new, parallel, tx format where fields are committed by merkle radix tree in an extensible and provable way. You'd then commit to that tree with a mandatory OP_RETURN output in the last txout, or with a new merkle root. Changing the tx format itself in a hard-fork is needlessly disruptive, and in this case, wastes opportunities for improvement. -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQFQBAEBCAA6BQJT4kzQMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhamzCAC+zRaXRodP63+ke3K+ Viapiepvk4uIOlqxqtMB2O0zWcyu2+xCJDiRPykK/6HLDBeFDEC9/dGK8++Lovl6 //qZ340LOPFlgT2kYy9E5h/yX469fhtsWhBCv2K47fWwkMS0S/0r4SQnCkbt2R2c 4dQjkoldhw6rNMBTUmwvhSlL30KsT/msWTZiX7DW/YjfOzezEJzy+mYyKp9Sk7ba 1fOiBXORk7mNOs7sTYTvje3sqEGpGTOLP08cY/RCEvl6bG8mHkPqwiojq+3biHFP RsoBVu1f5cbnU7Wq0gPNdVnQssnEQDadyTX8gT0Wze7PuVyaZT2mXFZBKzSHuLy2 sJKN =oPSo -END PGP SIGNATURE- -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
A fork is not necessarily required, if you are talking about information that deals primarily with pre-consensus mempool behavior. You can make a network TX with some information that is digitally signed, yet discarded before it reaches miners. On Wed, Aug 6, 2014 at 11:42 AM, Peter Todd p...@petertodd.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 6 August 2014 08:17:02 GMT-07:00, Christian Decker decker.christ...@gmail.com wrote: +1 for the new field, overloading fields with new meaning is definitely not a good idea. To add a new field the best way to do it is create a new, parallel, tx format where fields are committed by merkle radix tree in an extensible and provable way. You'd then commit to that tree with a mandatory OP_RETURN output in the last txout, or with a new merkle root. Changing the tx format itself in a hard-fork is needlessly disruptive, and in this case, wastes opportunities for improvement. -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQFQBAEBCAA6BQJT4kzQMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhamzCAC+zRaXRodP63+ke3K+ Viapiepvk4uIOlqxqtMB2O0zWcyu2+xCJDiRPykK/6HLDBeFDEC9/dGK8++Lovl6 //qZ340LOPFlgT2kYy9E5h/yX469fhtsWhBCv2K47fWwkMS0S/0r4SQnCkbt2R2c 4dQjkoldhw6rNMBTUmwvhSlL30KsT/msWTZiX7DW/YjfOzezEJzy+mYyKp9Sk7ba 1fOiBXORk7mNOs7sTYTvje3sqEGpGTOLP08cY/RCEvl6bG8mHkPqwiojq+3biHFP RsoBVu1f5cbnU7Wq0gPNdVnQssnEQDadyTX8gT0Wze7PuVyaZT2mXFZBKzSHuLy2 sJKN =oPSo -END PGP SIGNATURE- -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
Today we have first-eligible-height (nLockTime), and mempool expiration measured from this height would work for the goals being discussed, no fork or protocol rev. With first-eligible-height and last-eligible-height, creator could choose a lifetime shorter than the max, and in addition, lock the whole thing until some point in the future. On 8/6/2014 9:15 AM, Jeff Garzik wrote: A fork is not necessarily required, if you are talking about information that deals primarily with pre-consensus mempool behavior. You can make a network TX with some information that is digitally signed, yet discarded before it reaches miners. On Wed, Aug 6, 2014 at 11:42 AM, Peter Todd p...@petertodd.org mailto:p...@petertodd.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 6 August 2014 08:17:02 GMT-07:00, Christian Decker decker.christ...@gmail.com mailto:decker.christ...@gmail.com wrote: +1 for the new field, overloading fields with new meaning is definitely not a good idea. To add a new field the best way to do it is create a new, parallel, tx format where fields are committed by merkle radix tree in an extensible and provable way. You'd then commit to that tree with a mandatory OP_RETURN output in the last txout, or with a new merkle root. Changing the tx format itself in a hard-fork is needlessly disruptive, and in this case, wastes opportunities for improvement. -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQFQBAEBCAA6BQJT4kzQMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhamzCAC+zRaXRodP63+ke3K+ Viapiepvk4uIOlqxqtMB2O0zWcyu2+xCJDiRPykK/6HLDBeFDEC9/dGK8++Lovl6 //qZ340LOPFlgT2kYy9E5h/yX469fhtsWhBCv2K47fWwkMS0S/0r4SQnCkbt2R2c 4dQjkoldhw6rNMBTUmwvhSlL30KsT/msWTZiX7DW/YjfOzezEJzy+mYyKp9Sk7ba 1fOiBXORk7mNOs7sTYTvje3sqEGpGTOLP08cY/RCEvl6bG8mHkPqwiojq+3biHFP RsoBVu1f5cbnU7Wq0gPNdVnQssnEQDadyTX8gT0Wze7PuVyaZT2mXFZBKzSHuLy2 sJKN =oPSo -END PGP SIGNATURE- -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 6 August 2014 09:31:24 GMT-07:00, Mark Friedenbach m...@monetize.io wrote: I highly doubt that is the best approach. If this nExpiry field is a consensus rule, then the Merkle tree or the appropriate paths through needs to be included with the transaction as part of the network and on-disk data structures, so that proper validation can be done. This would be both more disruptive and less efficient than simply adding an nExpiry field to the transaction format, as we do in Freimarkets. The general case doesn't require transmission of any merkle data; it is derived from the tx data. Equally changing a data format is certainly: note how Freimarkets has no third-party library support because you've made it incompatible with the standard Bitcoin data structures. Merkle radix tree formatting OTOH is just a cryptographically committed extension of the tag-value concept seen in protobuf, among others. re: efficiency, we need fundamental improvements in efficiency, not little micro-optimisations everywhere done at high cost to maintainability. re: validation, note how the merkle radix tree meets that need by allowing the absence of data to be proven. If the field is pre-consensus (a mempool gentleman's agreement), then it has no business in the transaction structure at all and should be packaged in some sort of envelope container. It's also rather useless without consensus. Expiry is only useful if it is a guarantee, if not you might as well just implement tx replacement directly. -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQFQBAEBCAA6BQJT4mPZMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhcbKCACz/Qh3wm7ud9iwbvm3 Hzib36/fixk2++z6xlxh8G8afUaAe7ADCoz/TLK7RNIhUnr2hlsPO+Id2XvVBSm1 gXavj4iDxq8TpWsC8zPs5vyyY/dVwQ0RbidQFSpncmdW6EYVpIQp9nP3sSnBv1M8 c7BVidg708tc44uYiM9jeTzh6amP5yD0+G9FYYmy36BAQj8+4iD1ZCkiye1y5WUL 9MSN8LOxRFEwWQJmySXmJ1I9V81l1NSRQcQtfLVCzEIWLJXrZ0xwOq0SryEObg73 72NZKc2u8la3CPDoCG773ENbGHl+mGJW9M5ypV0s2RdkdZMgaFNnl/SBrWAcPd43 FkLr =OMOy -END PGP SIGNATURE- -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
That won't necessarily work through large reorgs. You don't want to create a situation where a miner cannot mine a previously mined transactions. On Wed, Aug 6, 2014 at 1:02 PM, Tom Harding t...@thinlink.com wrote: Today we have first-eligible-height (nLockTime), and mempool expiration measured from this height would work for the goals being discussed, no fork or protocol rev. With first-eligible-height and last-eligible-height, creator could choose a lifetime shorter than the max, and in addition, lock the whole thing until some point in the future. On 8/6/2014 9:15 AM, Jeff Garzik wrote: A fork is not necessarily required, if you are talking about information that deals primarily with pre-consensus mempool behavior. You can make a network TX with some information that is digitally signed, yet discarded before it reaches miners. On Wed, Aug 6, 2014 at 11:42 AM, Peter Todd p...@petertodd.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 6 August 2014 08:17:02 GMT-07:00, Christian Decker decker.christ...@gmail.com wrote: +1 for the new field, overloading fields with new meaning is definitely not a good idea. To add a new field the best way to do it is create a new, parallel, tx format where fields are committed by merkle radix tree in an extensible and provable way. You'd then commit to that tree with a mandatory OP_RETURN output in the last txout, or with a new merkle root. Changing the tx format itself in a hard-fork is needlessly disruptive, and in this case, wastes opportunities for improvement. -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQFQBAEBCAA6BQJT4kzQMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhamzCAC+zRaXRodP63+ke3K+ Viapiepvk4uIOlqxqtMB2O0zWcyu2+xCJDiRPykK/6HLDBeFDEC9/dGK8++Lovl6 //qZ340LOPFlgT2kYy9E5h/yX469fhtsWhBCv2K47fWwkMS0S/0r4SQnCkbt2R2c 4dQjkoldhw6rNMBTUmwvhSlL30KsT/msWTZiX7DW/YjfOzezEJzy+mYyKp9Sk7ba 1fOiBXORk7mNOs7sTYTvje3sqEGpGTOLP08cY/RCEvl6bG8mHkPqwiojq+3biHFP RsoBVu1f5cbnU7Wq0gPNdVnQssnEQDadyTX8gT0Wze7PuVyaZT2mXFZBKzSHuLy2 sJKN =oPSo -END PGP SIGNATURE- -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future.http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing listBitcoin-development@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On 08/06/2014 01:02 PM, Tom Harding wrote: With first-eligible-height and last-eligible-height, creator could choose a lifetime shorter than the max, and in addition, lock the whole thing until some point in the future. Note that this would be a massive, *massive* change that would completely break bitcoin output frangibility. Merchants would have to start demanding input history back to a certain depth in order to ensure they are not exposing themselves to undue reorg-expiry risk. There are useful applications of a consensus-enforced expiry, particularly within a private (signed block) side chain, and for that reason it is useful to have a discussion about the merits of an nExpiry field or BLOCK_HEIGHT / BLOCK_TIME opcode, and methods for achieving either. However I don't see this ever becoming part of the public bitcoin network. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On 08/06/2014 01:20 PM, Peter Todd wrote: The general case doesn't require transmission of any merkle data; it is derived from the tx data. How can that possibly be the case? The information is hidden behind the Merkle root in the transaction. The validator needs to know whether there is an expiry and what it is. What's it supposed to do, guess? -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 6 August 2014 10:21:56 GMT-07:00, Mark Friedenbach m...@monetize.io wrote: On 08/06/2014 01:02 PM, Tom Harding wrote: With first-eligible-height and last-eligible-height, creator could choose a lifetime shorter than the max, and in addition, lock the whole thing until some point in the future. Note that this would be a massive, *massive* change that would completely break bitcoin output frangibility. Merchants would have to start demanding input history back to a certain depth in order to ensure they are not exposing themselves to undue reorg-expiry risk. Bitcoin is already broken in that regard due to malleability, and more fundamentally, the existence of anyone-can-spend outputs, known private keys, SIGHASH_ANYONECANPAY, etc. In any case, reorg-doublespend risk is no different than reorg-expiry risk. -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQFQBAEBCAA6BQJT4mcdMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhSdiB/9no/fXR50Zej4l6Hyt gDvM9GWosGxZydQfplrUYzS9nLWTJgkjNYkrJk1OXPlkiNoHhlpCK6TuEL3DXBo8 txDBhp/xls7aFHELpPhP5iKrEj0J6fyMp9wKRVtUu0J+RhHY22v+iEQf//dGUX4v hQPwATubmnyeVd71TAKyW6zCPjoEh0IG19tRVvw/v7/qNTXHdSZTkSVzQa4GP2gr 2xVqXTeOycPKqIU+GaNI4aRAL2DUm1kW3jG/+h3BwnJNd5q+0ELpC6xDmkA6hkNz N6BFCtoghhKNH+FNsZKAzE9w8dYngZQbaA9vVdaR6SXzz9KuG526EymOF7e55IBJ FMu+ =ii2+ -END PGP SIGNATURE- -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 6 August 2014 10:30:11 GMT-07:00, Mark Friedenbach m...@monetize.io wrote: On 08/06/2014 01:20 PM, Peter Todd wrote: The general case doesn't require transmission of any merkle data; it is derived from the tx data. How can that possibly be the case? The information is hidden behind the Merkle root in the transaction. The validator needs to know whether there is an expiry and what it is. What's it supposed to do, guess? The general case is all committed information is included in the transaction; the merkle tree is a compatibility path, as well as an optimisation for lite clients and applications. You should read more about soft-forks; see the BIP. Remember that Bitcoin protocol development and deployment is not a centrally controlled activity. -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQFQBAEBCAA6BQJT4mgPMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhSYlCAC1ncAGQt53HKS+8/rq OG0RGrqE2l2/qCM/ybd9M8TkwxaI3NB5bqfIus8dB5MnyiTBFS3ooN54kNNOHtSX 2rEzPJphtOj46tk3nqe1QO3cbFJPjBCtxZff51DWZckhCiO2Iy1Br3fK3v55iscp 1jxyZnpfgUG/Ivfx+h6vkisucBXgXJ82d5vzvMIMxixh4v2+4/SAcSY6HCLIpxmV Z3l0NcGllnmWe5B6eftpWYUAREuoCNk/671jHmwu0cqk2u/Egrp776zxkEO1xivH d0EWjJmlDLmQ2hEhkpBq46ji/2m4EWPLqTW/EXf3RzwU8uCEldbxEe2tyZ0d6oBt NnTE =AhV7 -END PGP SIGNATURE- -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
It would make more sense to introduce a new script opcode that pushes the current block height onto the operand stack. Then you could implement arbitrary logic about which blocks the transaction can be valid in. This would require that the client revalidate all transactions in its mempool (really, only those making use of this opcode) whenever the chain tip changes. I have to say I like this idea, this would allow someone to prove they can't spend funds before a given date, and vice versa, prove that the funds can't ever be spent after a given date (and this is provably prunable, isn't it?). Of course, there are some risks associated with that, but nobody is forced to use it. flooding the network with unrelated high fee transactions in order to push a transaction out to where it can never be mined at all. This becomes increasingly expensive as the deadline is further away, so not very hard to mitigate. On Sat, Aug 2, 2014 at 1:36 AM, Tom Harding t...@thinlink.com wrote: On 7/31/2014 5:58 PM, Kaz Wesley wrote: 1. start setting nLockTime to the current height by default in newly created transactions (or slightly below the current height, for reorg-friendliness) Reorg-frendliness is the opposite of the rationale behind #2340, which proposes setting nLockTime at current-height + 1 to prevent fee-sniping reorgs... 2. once users have had some time to upgrade to clients that set nLockTime, start discouraging transactions without nLockTime -- possibly with a slightly higher fee required for relay 3. start rate-limiting relay of transactions without an nLockTime (maybe this alone could be used to achieve [2]) 4. add a new IsStandard rule rejecting transactions with an nLockTime more than N blocks behind the current tip (for some fixed value N, to be determined) One way to proceed is implement #3753 (mempool janitor) in such a way that transactions with nLockTime are allowed to live a bit longer in the mempool (say 500 blocks) than those without (72 hours). In other words, as a first step, just actually start expiring things from the mempool in bitcoin core, and leave any relay fee adjustments or rate limiting for later. The isStandard change would be a good complement to #3753, to avoid relaying a tx that will soon expire by the nLockTime rule anyway. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
Glad this was brought up. Transaction expiration is something that I have wanted to see happen in bitcoin for a long, long time. The user experience of unconfirming transactions setting around in limbo is just horrible. Bitcoin software by necessity has gotten better about attaching fees so this sort of behavior is uncommon, but that does not eliminate the problem. Of course, we cannot presume that a transaction will truly disappear -- The Internet Never Forgets -- but given a bit of mempool adjusting, we can achieve the next best thing: the majority of the network forgets the transaction and becomes willing to relay a respend of some or all of the inputs. This uses existing client logic where the client must rebroadcast a transaction until it is confirmed. In general, if a transaction has not made it into a block within 144*X blocks, there is _some_ reason it is getting rejected by the miners. The mempool janitor is a garbage collector design. This is inferior to the superblock model described at https://github.com/bitcoin/bitcoin/issues/3723 Other models can also achieve similar results. There are a lot of issues tied together here: transaction expiration, the desire to cap the mempool ram usage, scalability, DoS prevention, ... mempool ties a lot together. -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
A distinction there is that they can only become invalid via a conflict— replaced by another transaction authored by the prior signers. If no other transaction could be created (e.g. you're a multisigner and won't sign it again) then there is no such risk. You need to check transaction's dependencies up to a certain depth to know whether it is safe: If one of inputs depends on transaction which is signed by parties with unknown trustworthiness, then it isn't safe. It now introduces chance events (act of god) into the mix where they they didn't exist before. You need to check transaction's dependencies up to a certain depth to know whether it is safe: If one of inputs depends on transaction time-locked script (or other unrecognized script), then it isn't safe. Situation is identical, you might need several extra lines of code. I think it would matter only if we had deterministic, reliable mempool and reorganization behavior. But it's not something we can depend on. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
The user experience of unconfirming transactions setting around in limbo is just horrible. Bitcoin software by necessity has gotten better about attaching fees so this sort of behavior is uncommon, but that does not eliminate the problem. Yes, indeed. I suspect there's a quick hack that could make this problem a lot better though. I think I brought up this idea before, but can't quite remember. Anyway I'm willing to bet that if we analysed the data some more, we'd discover that most legitimate i.e. non-DoS unconfirmed transactions that sit around for ages are linked back to the block chain within two hops and not more. That is people send a transaction that uses up their coin age, and then immediately those coins are immediately respent again, but then those final new coins are not spent. On the other hand DoS attacks look like bouncing your coins around over and over forever, i.e. more than two or three hops back to the chain. So I wonder if making priority look back two or three transactions but not more would help real users a lot, whilst not opening up any significant new potential for DoS. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
I feel like a lot of this will be driven by implementation, and costs of changing the implementation. Additional look-backs are of course doable, but they incur some disk I/O costs. The fields available in memory for each mempool TX are uint256 tx_hash; // hash of next field CTransaction tx; int64_t nFee; // Cached to avoid expensive parent-transaction lookups size_t nTxSize; // ... and avoid recomputing tx size int64_t nTime; // Local time when entering the mempool double dPriority; // Priority when entering the mempool unsigned int nHeight; // Chain height when entering the mempool As a first pass, we may prune the mempool without additional db lookups quite easily based on time criteria. Or, additional in-memory indexes may be constructed to maintain hashes ordered by priority/fees. Those techniques seem likely to be attempted before resorting to looking back two or three TXs deep at coin age -- which I admit is an interesting metric. -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
In general, if a transaction has not made it into a block within 144*X blocks, there is _some_ reason it is getting rejected by the miners. This is the crux of my concern: relay policy and miner priorities will not necessarily always be in sync, and node resource management shouldn't rely on them being compatible. There are other solutions than transaction expiration; I think Gavin's idea from the block-squashing thread, in which miners explicitly provide information about their policies, would go a long way to address this. But even when mechanisms for reconciling nodes' expectations about miners' behavior with miners' actual behavior are available, it may be desirable to keep an expiry mechanism in place in case of glitches between node understanding of policy and actual miner policy. Any approach based on beginning a transaction expiry countdown when a transaction is received (as in mempool janitor) seems unviable to me: once a node has forgotten a transaction, it must be susceptible to reaccepting it; all the functionality of such an expiry mechanism depends on the network not containing any nodes with slightly different relay behavior from bitcoind. I could accidentally debilitate mempool janitors across the entire network if I set up two nodes to exchange mempools whenever they reconnected to each other, and restarted each frequently. That's why I think including information in the transaction itself, as with my nLockTime/IsStandard proposal, is necessary for transactions to reliably eventually die off from mempools. There's a modification I've been thinking about: allow a transaction's lifetime to be refreshed (even after expiry) by a child transaction, along the lines of child-pays-for-parent fee policy. This would eliminate the need to reuse a key to make a replacement for an expired transaction (when submitting the tx directly to a miner is not an option), as well as alleviating the potential inconvenience in cases like Peter brought up, where nLockTime is used for exchanged locked transactions as part of a multi-transaction contract. With child-refreshes-parent, a transaction's receivers and senders would have the ability to keep trying to get their payment confirmed, but anyone on the network can't just keep all transactions alive. On Tue, Aug 5, 2014 at 10:48 AM, Jeff Garzik jgar...@bitpay.com wrote: Glad this was brought up. Transaction expiration is something that I have wanted to see happen in bitcoin for a long, long time. The user experience of unconfirming transactions setting around in limbo is just horrible. Bitcoin software by necessity has gotten better about attaching fees so this sort of behavior is uncommon, but that does not eliminate the problem. Of course, we cannot presume that a transaction will truly disappear -- The Internet Never Forgets -- but given a bit of mempool adjusting, we can achieve the next best thing: the majority of the network forgets the transaction and becomes willing to relay a respend of some or all of the inputs. This uses existing client logic where the client must rebroadcast a transaction until it is confirmed. In general, if a transaction has not made it into a block within 144*X blocks, there is _some_ reason it is getting rejected by the miners. The mempool janitor is a garbage collector design. This is inferior to the superblock model described at https://github.com/bitcoin/bitcoin/issues/3723 Other models can also achieve similar results. There are a lot of issues tied together here: transaction expiration, the desire to cap the mempool ram usage, scalability, DoS prevention, ... mempool ties a lot together. -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Tue, Aug 5, 2014 at 3:10 PM, Kaz Wesley kezi...@gmail.com wrote: Any approach based on beginning a transaction expiry countdown when a transaction is received (as in mempool janitor) seems unviable to me: ... That's why I think including information in the transaction itself, as with my nLockTime/IsStandard proposal, is necessary for transactions to reliably eventually die off from mempools. reliably die off from mempools leads into the land of tightly synchronizing memory pools across the network which is a problem of... large scope and much debate. :) For the moment, simply capping the mempool's size at each local node is a much more reachable goal. Capping, then, implies some culling policy. In general, bitcoind Tx mempool size is rather open ended, and that needs sorting out. -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On 8/5/2014 12:10 PM, Kaz Wesley wrote: Any approach based on beginning a transaction expiry countdown when a transaction is received (as in mempool janitor) seems unviable to me: once a node has forgotten a transaction, it must be susceptible to reaccepting it; It's hard to argue with that logic. If nLockTime is used for expiration, transaction creator can't lie to help tx live longer without pushing initial confirmation eligibility into the future. Very pretty. It would also enable fill or kill transactions with a backdated nLockTime, which must be confirmed in a few blocks, or start vanishing from mempools. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On 7/31/2014 5:58 PM, Kaz Wesley wrote: 1. start setting nLockTime to the current height by default in newly created transactions (or slightly below the current height, for reorg-friendliness) Reorg-frendliness is the opposite of the rationale behind #2340, which proposes setting nLockTime at current-height + 1 to prevent fee-sniping reorgs... 2. once users have had some time to upgrade to clients that set nLockTime, start discouraging transactions without nLockTime -- possibly with a slightly higher fee required for relay 3. start rate-limiting relay of transactions without an nLockTime (maybe this alone could be used to achieve [2]) 4. add a new IsStandard rule rejecting transactions with an nLockTime more than N blocks behind the current tip (for some fixed value N, to be determined) One way to proceed is implement #3753 (mempool janitor) in such a way that transactions with nLockTime are allowed to live a bit longer in the mempool (say 500 blocks) than those without (72 hours). In other words, as a first step, just actually start expiring things from the mempool in bitcoin core, and leave any relay fee adjustments or rate limiting for later. The isStandard change would be a good complement to #3753, to avoid relaying a tx that will soon expire by the nLockTime rule anyway. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thu, Jul 31, 2014 at 05:58:23PM -0700, Kaz Wesley wrote: I have a proposal for a way to add finite and predictable lifespans to transactions in mempools: we d̶e̶s̶t̶r̶o̶y̶ ̶t̶h̶e̶ ̶r̶e̶s̶u̶r̶r̶e̶c̶t̶i̶o̶n̶ ̶h̶u̶b̶ use nLockTime and a new standardness rule. It could be done in stages, would not necessarily require even a soft fork, and does not cause problems with reorgs like the proposal in #3509: Anything that changes the semantics of nLockTime will do harm to existing and future applications that make use of nLockTime for things like refund transactions. In any case, why do transactions need finite lifespans in mempools? If you want to double-spend them with higher fees, then implement replace-by-fee. In any case, lifetimes will never be deterministic as not everyone runs the same software. Transactions would stop being relayed and drop out of mempools a fixed number of blocks from their creation; once that window had passed, the sender's wallet could begin to expect the transaction would not be confirmed. In case a reorg displaces a transaction until after its expiry height, a miner can still put it back in the blockchain; the expiry height is just a relay rule. Also, a user who needed to get their original expired transaction confirmed could still do so by submitting it directly to a miner with suitable policies. ...in which case someone will circumvent this IsStandard() rule by submitting expired transactions directly to miners with suitable policies. signature.asc Description: Digital signature -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thu, Jul 31, 2014 at 6:06 PM, Peter Todd p...@petertodd.org wrote: Anything that changes the semantics of nLockTime will do harm to existing and future applications that make use of nLockTime for things like refund transactions. I think this would be compatible with most uses of nLockTime -- e.g., at the time a refund transaction becomes broadcastable, its beneficiary would usually have no reason to wait for a long time before broadcasting it; if they did so (probably because they weren't online to redeem the refund), they'd need to use the submit-directly-to-miner option, but they wouldn't lose their refund. In any case, why do transactions need finite lifespans in mempools? If you want to double-spend them with higher fees, then implement replace-by-fee. Perpetuating transactions that have been in mempools for a long time and are not being confirmed has been cited as a reason for nodes not to exchange mempools (#3721, #1833, #3722); it's been implied that it would be desirable for wallets to wait until a transaction had had a chance to be accepted before double-spending with a higher fee (#3722); and an unconfirmed transaction-age-based policy for preventing mempool accumulation has been advocated (#3753, #3722) [I hope my summarization is not misrepresenting anyone's opinions here; please see the arguments made in the actual comments on the bugs]. These discussions are mostly fairly old, but I don't know of any changes that have been made that provide alternative answers to these concerns mentioned by at least three different developers. In any case, lifetimes will never be deterministic as not everyone runs the same software. That's true, but none of the benefits of these changes require the policy to be unanimous; most of the effect could be provided by most of the network following these rules. Transactions would stop being relayed and drop out of mempools a fixed number of blocks from their creation; once that window had passed, the sender's wallet could begin to expect the transaction would not be confirmed. In case a reorg displaces a transaction until after its expiry height, a miner can still put it back in the blockchain; the expiry height is just a relay rule. Also, a user who needed to get their original expired transaction confirmed could still do so by submitting it directly to a miner with suitable policies. ...in which case someone will circumvent this IsStandard() rule by submitting expired transactions directly to miners with suitable policies. Yes, that is a feature. None of the benefits of transaction expiration rely on expiration being final, and any possible downsides of expiration are largely mitigated by the option still being available to get expired transactions mined. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
It would make more sense to introduce a new script opcode that pushes the current block height onto the operand stack. Then you could implement arbitrary logic about which blocks the transaction can be valid in. This would require that the client revalidate all transactions in its mempool (really, only those making use of this opcode) whenever the chain tip changes. On Thursday, 31 July 2014, at 5:58 pm, Kaz Wesley wrote: There is currently little in place for managing transaction lifetime in the network's mempools (see discussion in github in #3722 mempool transaction expiration, and it seems to be a major factor blocking some mempool exchange, see #1833/1918, #3721). Expiry per-node a certain amount of wall time after receipt has been proposed, but that's a fragile mechanism -- a single node could keep all relayable transactions alive forever by remembering transactions until most nodes have dropped them and then releasing them back into the wild. I have a proposal for a way to add finite and predictable lifespans to transactions in mempools: we d̶e̶s̶t̶r̶o̶y̶ ̶t̶h̶e̶ ̶r̶e̶s̶u̶r̶r̶e̶c̶t̶i̶o̶n̶ ̶h̶u̶b̶ use nLockTime and a new standardness rule. It could be done in stages, would not necessarily require even a soft fork, and does not cause problems with reorgs like the proposal in #3509: 1. start setting nLockTime to the current height by default in newly created transactions (or slightly below the current height, for reorg-friendliness) 2. once users have had some time to upgrade to clients that set nLockTime, start discouraging transactions without nLockTime -- possibly with a slightly higher fee required for relay 3. start rate-limiting relay of transactions without an nLockTime (maybe this alone could be used to achieve [2]) 4. add a new IsStandard rule rejecting transactions with an nLockTime more than N blocks behind the current tip (for some fixed value N, to be determined) Transactions would stop being relayed and drop out of mempools a fixed number of blocks from their creation; once that window had passed, the sender's wallet could begin to expect the transaction would not be confirmed. In case a reorg displaces a transaction until after its expiry height, a miner can still put it back in the blockchain; the expiry height is just a relay rule. Also, a user who needed to get their original expired transaction confirmed could still do so by submitting it directly to a miner with suitable policies. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thu, Jul 31, 2014 at 6:38 PM, Matt Whitlock b...@mattwhitlock.name wrote: It would make more sense to introduce a new script opcode that pushes the current block height onto the operand stack. Then you could implement arbitrary logic about which blocks the transaction can be valid in. This would require that the client revalidate all transactions in its mempool (really, only those making use of this opcode) whenever the chain tip changes. Transactions that become invalid later are have pretty severe consequences because they might mean that completely in an absence of fraud transactions are forever precluded due to a otherwise harmless reorg. While there may be uses for that, the resulting outputs should be considered differently fungible— like coinbases which are immature— and should probably be only used with great caution... not as a mechanism for ordinary transactions. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thursday, 31 July 2014, at 7:28 pm, Gregory Maxwell wrote: On Thu, Jul 31, 2014 at 6:38 PM, Matt Whitlock b...@mattwhitlock.name wrote: It would make more sense to introduce a new script opcode that pushes the current block height onto the operand stack. Then you could implement arbitrary logic about which blocks the transaction can be valid in. This would require that the client revalidate all transactions in its mempool (really, only those making use of this opcode) whenever the chain tip changes. Transactions that become invalid later are have pretty severe consequences because they might mean that completely in an absence of fraud transactions are forever precluded due to a otherwise harmless reorg. I understand what you're saying, but I don't understand why it's a problem. Transactions shouldn't be considered final until a reasonable number of confirmations anyway, so the possibility that an accepted transaction could become invalid due to a chain reorganization is not a new danger. Ordinary transactions can similarly become invalid due to chain reorganizations, due to inputs already having been spent in the new branch. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] deterministic transaction expiration
On Thu, Jul 31, 2014 at 8:26 PM, Matt Whitlock b...@mattwhitlock.name wrote: I understand what you're saying, but I don't understand why it's a problem. Transactions shouldn't be considered final until a reasonable number of confirmations anyway, so the possibility that an accepted transaction could become invalid due to a chain reorganization is not a new danger. Ordinary transactions can similarly become invalid due to chain reorganizations, due to inputs already having been spent in the new branch. A distinction there is that they can only become invalid via a conflict— replaced by another transaction authored by the prior signers. If no other transaction could be created (e.g. you're a multisigner and won't sign it again) then there is no such risk. It now introduces chance events (act of god) into the mix where they they didn't exist before. Basically it takes was what is a very loose one way coupling and makes it much tighter. I'm sure if you spend a bit thinking you can come up with some more corner cases that it would expose— e.g. flooding the network with unrelated high fee transactions in order to push a transaction out to where it can never be mined at all. -- Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
