Re: [Bitcoin-development] deterministic transaction expiration

2014-08-08 Thread Tom Harding
Having explored more drastic approaches, it looks like Kaz' basic idea 
stands well.  His #1...

 1. start setting nLockTime to the current height by default in newly
 created transactions (or slightly below the current height, for
 reorg-friendliness)

is already implemented in bitcoin-qt #2340, and a final call on 
merging it was already sent to this list.  After some thought I agree 
with its policy of eventually setting nLockTime at current-height + 1 by 
default.  This is the best reasonably expected height of any tx 
created right now.  It discourages fee-sniping, and if a reorg happens 
anyway, it won't actually delay inclusion of tx beyond the reasonable 
expectation sans reorg.

However right now, #2340 takes a very cautious approach and sets to 
current-height - 10 by default, with randomness to mitigate worries 
about loss of privacy.

Kaz' #2, #3 and #4 are future actions.  #4 only goes most of the way ...

 4. add a new IsStandard rule rejecting transactions with an nLockTime
 more than N blocks behind the current tip (for some fixed value N, to
 be determined)

... a janitor mechanism is desirable to purge mempool of txes more than 
N behind current-height.

Nodes dropping a tx N blocks after they became eligible to be mined (the 
meaning of nLockTime) makes sense.  It is not an overloading or new use 
for nLockTime, but a logical extension of it.  As Kaz pointed out, this 
solves a big problem with expiring by locally measured age: 
unintentional resurrection.


--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-08 Thread Jeff Garzik
On Fri, Aug 8, 2014 at 1:38 PM, Tom Harding t...@thinlink.com wrote:
 4. add a new IsStandard rule rejecting transactions with an nLockTime
 more than N blocks behind the current tip (for some fixed value N, to
 be determined)

It cannot be assumed that transaction creation time and transaction
publish-to-outside-world time are the same, even though they often
are.

-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.  https://bitpay.com/

--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-08 Thread Kaz Wesley
A new network tx field would have the same problem, right?

With a child-refreshes-parent policy, someone wishing to redeem a
transaction that has passed its relay window without being confirmed could
still do so.
On Aug 8, 2014 11:16 AM, Jeff Garzik jgar...@bitpay.com wrote:

 On Fri, Aug 8, 2014 at 1:38 PM, Tom Harding t...@thinlink.com wrote:
  4. add a new IsStandard rule rejecting transactions with an nLockTime
  more than N blocks behind the current tip (for some fixed value N, to
  be determined)

 It cannot be assumed that transaction creation time and transaction
 publish-to-outside-world time are the same, even though they often
 are.

 --
 Jeff Garzik
 Bitcoin core developer and open source evangelist
 BitPay, Inc.  https://bitpay.com/


 --
 Want fast and easy access to all the code in your enterprise? Index and
 search up to 200,000 lines of code with a free copy of Black Duck
 Code Sight - the same software that powers the world's largest code
 search on Ohloh, the Black Duck Open Hub! Try it now.
 http://p.sf.net/sfu/bds
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development

--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Tom Harding


How is eventual expiration of a tx that started life with an nLockTime 
in the future breaking, any more than any other tx expiring?



On 8/6/2014 6:54 AM, Mike Hearn wrote:
We could however introduce a new field in a new tx version. We know we 
need to rev the format at some point anyway.



On Wed, Aug 6, 2014 at 2:55 PM, Jeff Garzik jgar...@bitpay.com 
mailto:jgar...@bitpay.com wrote:


 ...and existing users and uses of nLockTime suddenly become
worthless, breaking payment channel refunds and other active uses
of nLockTime.

You cannot assume the user is around to rewrite their nLockTime,
if it fails to be confirmed before some arbitrary deadline being set.



On Wed, Aug 6, 2014 at 12:01 AM, Tom Harding t...@thinlink.com
mailto:t...@thinlink.com wrote:

...




If nLockTime is used for expiration, transaction creator can't
lie to
help tx live longer without pushing initial confirmation
eligibility
into the future.  Very pretty.  It would also enable fill or
kill
transactions with a backdated nLockTime, which must be
confirmed in a
few blocks, or start vanishing from mempools.



--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Jeff Garzik
 ...because nLockTime is the exact opposite of expiration.  A locked TX
begins life invalid, and becomes valid (not-expired) after nLockTime passes.

A new field containing expiration time would work.



On Wed, Aug 6, 2014 at 10:44 AM, Tom Harding t...@thinlink.com wrote:


 How is eventual expiration of a tx that started life with an nLockTime in
 the future breaking, any more than any other tx expiring?



 On 8/6/2014 6:54 AM, Mike Hearn wrote:

 We could however introduce a new field in a new tx version. We know we
 need to rev the format at some point anyway.


 On Wed, Aug 6, 2014 at 2:55 PM, Jeff Garzik jgar...@bitpay.com wrote:

  ...and existing users and uses of nLockTime suddenly become worthless,
 breaking payment channel refunds and other active uses of nLockTime.

 You cannot assume the user is around to rewrite their nLockTime, if it
 fails to be confirmed before some arbitrary deadline being set.



 On Wed, Aug 6, 2014 at 12:01 AM, Tom Harding t...@thinlink.com wrote:

 ...


  If nLockTime is used for expiration, transaction creator can't lie to
 help tx live longer without pushing initial confirmation eligibility
 into the future.  Very pretty.  It would also enable fill or kill
 transactions with a backdated nLockTime, which must be confirmed in a
 few blocks, or start vanishing from mempools.





-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.  https://bitpay.com/
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Christian Decker
+1 for the new field, overloading fields with new meaning is definitely not
a good idea.

Something like nExpireAt with a block height sounds reasonable to me, but
we need to document that the usual caveats with blockchain reorgs apply.


On Wed, Aug 6, 2014 at 4:08 PM, Jeff Garzik jgar...@bitpay.com wrote:

  ...because nLockTime is the exact opposite of expiration.  A locked TX
 begins life invalid, and becomes valid (not-expired) after nLockTime passes.

 A new field containing expiration time would work.



 On Wed, Aug 6, 2014 at 10:44 AM, Tom Harding t...@thinlink.com wrote:


 How is eventual expiration of a tx that started life with an nLockTime in
 the future breaking, any more than any other tx expiring?



 On 8/6/2014 6:54 AM, Mike Hearn wrote:

 We could however introduce a new field in a new tx version. We know we
 need to rev the format at some point anyway.


 On Wed, Aug 6, 2014 at 2:55 PM, Jeff Garzik jgar...@bitpay.com wrote:

  ...and existing users and uses of nLockTime suddenly become worthless,
 breaking payment channel refunds and other active uses of nLockTime.

 You cannot assume the user is around to rewrite their nLockTime, if it
 fails to be confirmed before some arbitrary deadline being set.



 On Wed, Aug 6, 2014 at 12:01 AM, Tom Harding t...@thinlink.com wrote:

 ...


  If nLockTime is used for expiration, transaction creator can't lie
 to
 help tx live longer without pushing initial confirmation eligibility
 into the future.  Very pretty.  It would also enable fill or kill
 transactions with a backdated nLockTime, which must be confirmed in a
 few blocks, or start vanishing from mempools.





 --
 Jeff Garzik
 Bitcoin core developer and open source evangelist
 BitPay, Inc.  https://bitpay.com/


 --
 Infragistics Professional
 Build stunning WinForms apps today!
 Reboot your WinForms applications with our WinForms controls.
 Build a bridge from your legacy apps to the future.

 http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256



On 6 August 2014 08:17:02 GMT-07:00, Christian Decker 
decker.christ...@gmail.com wrote:
+1 for the new field, overloading fields with new meaning is definitely
not
a good idea.

To add a new field the best way to do it is create a new, parallel, tx format 
where fields are committed by merkle radix tree in an extensible and provable 
way. You'd then commit to that tree with a mandatory OP_RETURN output in the 
last txout, or with a new merkle root.

Changing the tx format itself in a hard-fork is needlessly disruptive, and in 
this case, wastes opportunities for improvement.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCAA6BQJT4kzQMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhamzCAC+zRaXRodP63+ke3K+
Viapiepvk4uIOlqxqtMB2O0zWcyu2+xCJDiRPykK/6HLDBeFDEC9/dGK8++Lovl6
//qZ340LOPFlgT2kYy9E5h/yX469fhtsWhBCv2K47fWwkMS0S/0r4SQnCkbt2R2c
4dQjkoldhw6rNMBTUmwvhSlL30KsT/msWTZiX7DW/YjfOzezEJzy+mYyKp9Sk7ba
1fOiBXORk7mNOs7sTYTvje3sqEGpGTOLP08cY/RCEvl6bG8mHkPqwiojq+3biHFP
RsoBVu1f5cbnU7Wq0gPNdVnQssnEQDadyTX8gT0Wze7PuVyaZT2mXFZBKzSHuLy2
sJKN
=oPSo
-END PGP SIGNATURE-


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Jeff Garzik
A fork is not necessarily required, if you are talking about information
that deals primarily with pre-consensus mempool behavior.  You can make a
network TX with some information that is digitally signed, yet discarded
before it reaches miners.


On Wed, Aug 6, 2014 at 11:42 AM, Peter Todd p...@petertodd.org wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256



 On 6 August 2014 08:17:02 GMT-07:00, Christian Decker 
 decker.christ...@gmail.com wrote:
 +1 for the new field, overloading fields with new meaning is definitely
 not
 a good idea.

 To add a new field the best way to do it is create a new, parallel, tx
 format where fields are committed by merkle radix tree in an extensible and
 provable way. You'd then commit to that tree with a mandatory OP_RETURN
 output in the last txout, or with a new merkle root.

 Changing the tx format itself in a hard-fork is needlessly disruptive, and
 in this case, wastes opportunities for improvement.
 -BEGIN PGP SIGNATURE-
 Version: APG v1.1.1

 iQFQBAEBCAA6BQJT4kzQMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhamzCAC+zRaXRodP63+ke3K+
 Viapiepvk4uIOlqxqtMB2O0zWcyu2+xCJDiRPykK/6HLDBeFDEC9/dGK8++Lovl6
 //qZ340LOPFlgT2kYy9E5h/yX469fhtsWhBCv2K47fWwkMS0S/0r4SQnCkbt2R2c
 4dQjkoldhw6rNMBTUmwvhSlL30KsT/msWTZiX7DW/YjfOzezEJzy+mYyKp9Sk7ba
 1fOiBXORk7mNOs7sTYTvje3sqEGpGTOLP08cY/RCEvl6bG8mHkPqwiojq+3biHFP
 RsoBVu1f5cbnU7Wq0gPNdVnQssnEQDadyTX8gT0Wze7PuVyaZT2mXFZBKzSHuLy2
 sJKN
 =oPSo
 -END PGP SIGNATURE-




-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.  https://bitpay.com/
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Mark Friedenbach
On 08/06/2014 11:42 AM, Peter Todd wrote:
 On 6 August 2014 08:17:02 GMT-07:00, Christian Decker
 decker.christ...@gmail.com wrote:
 +1 for the new field, overloading fields with new meaning is
 definitely not a good idea.
 
 To add a new field the best way to do it is create a new, parallel,
 tx format where fields are committed by merkle radix tree in an
 extensible and provable way. You'd then commit to that tree with a
 mandatory OP_RETURN output in the last txout, or with a new merkle
 root.
 
 Changing the tx format itself in a hard-fork is needlessly
 disruptive, and in this case, wastes opportunities for improvement.

I highly doubt that is the best approach.

If this nExpiry field is a consensus rule, then the Merkle tree or the
appropriate paths through needs to be included with the transaction as
part of the network and on-disk data structures, so that proper
validation can be done. This would be both more disruptive and less
efficient than simply adding an nExpiry field to the transaction format,
as we do in Freimarkets.

If the field is pre-consensus (a mempool gentleman's agreement), then it
has no business in the transaction structure at all and should be
packaged in some sort of envelope container.

--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Tom Harding


Today we have first-eligible-height (nLockTime), and mempool expiration 
measured from this height would work for the goals being discussed, no 
fork or protocol rev.


With first-eligible-height and last-eligible-height, creator could 
choose a lifetime shorter than the max,  and in addition, lock the whole 
thing until some point in the future.



On 8/6/2014 9:15 AM, Jeff Garzik wrote:
A fork is not necessarily required, if you are talking about 
information that deals primarily with pre-consensus mempool behavior.  
You can make a network TX with some information that is digitally 
signed, yet discarded before it reaches miners.



On Wed, Aug 6, 2014 at 11:42 AM, Peter Todd p...@petertodd.org 
mailto:p...@petertodd.org wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256



On 6 August 2014 08:17:02 GMT-07:00, Christian Decker
decker.christ...@gmail.com mailto:decker.christ...@gmail.com
wrote:
+1 for the new field, overloading fields with new meaning is
definitely
not
a good idea.

To add a new field the best way to do it is create a new,
parallel, tx format where fields are committed by merkle radix
tree in an extensible and provable way. You'd then commit to that
tree with a mandatory OP_RETURN output in the last txout, or with
a new merkle root.

Changing the tx format itself in a hard-fork is needlessly
disruptive, and in this case, wastes opportunities for improvement.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCAA6BQJT4kzQMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhamzCAC+zRaXRodP63+ke3K+
Viapiepvk4uIOlqxqtMB2O0zWcyu2+xCJDiRPykK/6HLDBeFDEC9/dGK8++Lovl6
//qZ340LOPFlgT2kYy9E5h/yX469fhtsWhBCv2K47fWwkMS0S/0r4SQnCkbt2R2c
4dQjkoldhw6rNMBTUmwvhSlL30KsT/msWTZiX7DW/YjfOzezEJzy+mYyKp9Sk7ba
1fOiBXORk7mNOs7sTYTvje3sqEGpGTOLP08cY/RCEvl6bG8mHkPqwiojq+3biHFP
RsoBVu1f5cbnU7Wq0gPNdVnQssnEQDadyTX8gT0Wze7PuVyaZT2mXFZBKzSHuLy2
sJKN
=oPSo
-END PGP SIGNATURE-




--
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc. https://bitpay.com/


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk


___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256



On 6 August 2014 09:31:24 GMT-07:00, Mark Friedenbach m...@monetize.io wrote:
I highly doubt that is the best approach.

If this nExpiry field is a consensus rule, then the Merkle tree or the
appropriate paths through needs to be included with the transaction as
part of the network and on-disk data structures, so that proper
validation can be done. This would be both more disruptive and less
efficient than simply adding an nExpiry field to the transaction
format,
as we do in Freimarkets.

The general case doesn't require transmission of any merkle data; it is derived 
from the tx data. Equally changing a data format is certainly: note how 
Freimarkets has no third-party library support because you've made it 
incompatible with the standard Bitcoin data structures. Merkle radix tree 
formatting OTOH is just a cryptographically committed extension of the 
tag-value concept seen in protobuf, among others.

re: efficiency, we need fundamental improvements in efficiency, not little 
micro-optimisations everywhere done at high cost to maintainability.

re: validation, note how the merkle radix tree meets that need by allowing the 
absence of data to be proven.

If the field is pre-consensus (a mempool gentleman's agreement), then
it
has no business in the transaction structure at all and should be
packaged in some sort of envelope container.

It's also rather useless without consensus. Expiry is only useful if it is a 
guarantee, if not you might as well just implement tx replacement directly.

-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCAA6BQJT4mPZMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhcbKCACz/Qh3wm7ud9iwbvm3
Hzib36/fixk2++z6xlxh8G8afUaAe7ADCoz/TLK7RNIhUnr2hlsPO+Id2XvVBSm1
gXavj4iDxq8TpWsC8zPs5vyyY/dVwQ0RbidQFSpncmdW6EYVpIQp9nP3sSnBv1M8
c7BVidg708tc44uYiM9jeTzh6amP5yD0+G9FYYmy36BAQj8+4iD1ZCkiye1y5WUL
9MSN8LOxRFEwWQJmySXmJ1I9V81l1NSRQcQtfLVCzEIWLJXrZ0xwOq0SryEObg73
72NZKc2u8la3CPDoCG773ENbGHl+mGJW9M5ypV0s2RdkdZMgaFNnl/SBrWAcPd43
FkLr
=OMOy
-END PGP SIGNATURE-


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Jeff Garzik
That won't necessarily work through large reorgs.  You don't want to create
a situation where a miner cannot mine a previously mined transactions.



On Wed, Aug 6, 2014 at 1:02 PM, Tom Harding t...@thinlink.com wrote:


 Today we have first-eligible-height (nLockTime), and mempool expiration
 measured from this height would work for the goals being discussed, no fork
 or protocol rev.

 With first-eligible-height and last-eligible-height, creator could choose
 a lifetime shorter than the max,  and in addition, lock the whole thing
 until some point in the future.



 On 8/6/2014 9:15 AM, Jeff Garzik wrote:

 A fork is not necessarily required, if you are talking about information
 that deals primarily with pre-consensus mempool behavior.  You can make a
 network TX with some information that is digitally signed, yet discarded
 before it reaches miners.


 On Wed, Aug 6, 2014 at 11:42 AM, Peter Todd p...@petertodd.org wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256



 On 6 August 2014 08:17:02 GMT-07:00, Christian Decker 
 decker.christ...@gmail.com wrote:
 +1 for the new field, overloading fields with new meaning is definitely
 not
 a good idea.

  To add a new field the best way to do it is create a new, parallel, tx
 format where fields are committed by merkle radix tree in an extensible and
 provable way. You'd then commit to that tree with a mandatory OP_RETURN
 output in the last txout, or with a new merkle root.

 Changing the tx format itself in a hard-fork is needlessly disruptive,
 and in this case, wastes opportunities for improvement.
 -BEGIN PGP SIGNATURE-
 Version: APG v1.1.1

 iQFQBAEBCAA6BQJT4kzQMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhamzCAC+zRaXRodP63+ke3K+
 Viapiepvk4uIOlqxqtMB2O0zWcyu2+xCJDiRPykK/6HLDBeFDEC9/dGK8++Lovl6
 //qZ340LOPFlgT2kYy9E5h/yX469fhtsWhBCv2K47fWwkMS0S/0r4SQnCkbt2R2c
 4dQjkoldhw6rNMBTUmwvhSlL30KsT/msWTZiX7DW/YjfOzezEJzy+mYyKp9Sk7ba
 1fOiBXORk7mNOs7sTYTvje3sqEGpGTOLP08cY/RCEvl6bG8mHkPqwiojq+3biHFP
 RsoBVu1f5cbnU7Wq0gPNdVnQssnEQDadyTX8gT0Wze7PuVyaZT2mXFZBKzSHuLy2
 sJKN
 =oPSo
 -END PGP SIGNATURE-




 --
 Jeff Garzik
 Bitcoin core developer and open source evangelist
 BitPay, Inc.  https://bitpay.com/


 --
 Infragistics Professional
 Build stunning WinForms apps today!
 Reboot your WinForms applications with our WinForms controls.
 Build a bridge from your legacy apps to the 
 future.http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk



 ___
 Bitcoin-development mailing 
 listBitcoin-development@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bitcoin-development




 --
 Infragistics Professional
 Build stunning WinForms apps today!
 Reboot your WinForms applications with our WinForms controls.
 Build a bridge from your legacy apps to the future.

 http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development




-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.  https://bitpay.com/
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Mark Friedenbach
On 08/06/2014 01:02 PM, Tom Harding wrote:
 With first-eligible-height and last-eligible-height, creator could
 choose a lifetime shorter than the max,  and in addition, lock the whole
 thing until some point in the future.

Note that this would be a massive, *massive* change that would
completely break bitcoin output frangibility. Merchants would have to
start demanding input history back to a certain depth in order to ensure
they are not exposing themselves to undue reorg-expiry risk.

There are useful applications of a consensus-enforced expiry,
particularly within a private (signed block) side chain, and for that
reason it is useful to have a discussion about the merits of an nExpiry
field or BLOCK_HEIGHT / BLOCK_TIME opcode, and methods for achieving
either. However I don't see this ever becoming part of the public
bitcoin network.

--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Mark Friedenbach
On 08/06/2014 01:20 PM, Peter Todd wrote:
 The general case doesn't require transmission of any merkle data; it
 is derived from the tx data.

How can that possibly be the case? The information is hidden behind the
Merkle root in the transaction. The validator needs to know whether
there is an expiry and what it is. What's it supposed to do, guess?

--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256



On 6 August 2014 10:21:56 GMT-07:00, Mark Friedenbach m...@monetize.io wrote:
On 08/06/2014 01:02 PM, Tom Harding wrote:
 With first-eligible-height and last-eligible-height, creator could
 choose a lifetime shorter than the max,  and in addition, lock the
whole
 thing until some point in the future.

Note that this would be a massive, *massive* change that would
completely break bitcoin output frangibility. Merchants would have to
start demanding input history back to a certain depth in order to
ensure
they are not exposing themselves to undue reorg-expiry risk.

Bitcoin is already broken in that regard due to malleability, and more 
fundamentally, the existence of anyone-can-spend outputs, known private keys, 
SIGHASH_ANYONECANPAY, etc.

In any case, reorg-doublespend risk is no different than reorg-expiry risk.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCAA6BQJT4mcdMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhSdiB/9no/fXR50Zej4l6Hyt
gDvM9GWosGxZydQfplrUYzS9nLWTJgkjNYkrJk1OXPlkiNoHhlpCK6TuEL3DXBo8
txDBhp/xls7aFHELpPhP5iKrEj0J6fyMp9wKRVtUu0J+RhHY22v+iEQf//dGUX4v
hQPwATubmnyeVd71TAKyW6zCPjoEh0IG19tRVvw/v7/qNTXHdSZTkSVzQa4GP2gr
2xVqXTeOycPKqIU+GaNI4aRAL2DUm1kW3jG/+h3BwnJNd5q+0ELpC6xDmkA6hkNz
N6BFCtoghhKNH+FNsZKAzE9w8dYngZQbaA9vVdaR6SXzz9KuG526EymOF7e55IBJ
FMu+
=ii2+
-END PGP SIGNATURE-


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-06 Thread Peter Todd
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256



On 6 August 2014 10:30:11 GMT-07:00, Mark Friedenbach m...@monetize.io wrote:
On 08/06/2014 01:20 PM, Peter Todd wrote:
 The general case doesn't require transmission of any merkle data; it
 is derived from the tx data.

How can that possibly be the case? The information is hidden behind the
Merkle root in the transaction. The validator needs to know whether
there is an expiry and what it is. What's it supposed to do, guess?

The general case is all committed information is included in the transaction; 
the merkle tree is a compatibility path, as well as an optimisation for lite 
clients and applications.

You should read more about soft-forks; see the BIP. Remember that Bitcoin 
protocol development and deployment is not a centrally controlled activity.
-BEGIN PGP SIGNATURE-
Version: APG v1.1.1

iQFQBAEBCAA6BQJT4mgPMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhSYlCAC1ncAGQt53HKS+8/rq
OG0RGrqE2l2/qCM/ybd9M8TkwxaI3NB5bqfIus8dB5MnyiTBFS3ooN54kNNOHtSX
2rEzPJphtOj46tk3nqe1QO3cbFJPjBCtxZff51DWZckhCiO2Iy1Br3fK3v55iscp
1jxyZnpfgUG/Ivfx+h6vkisucBXgXJ82d5vzvMIMxixh4v2+4/SAcSY6HCLIpxmV
Z3l0NcGllnmWe5B6eftpWYUAREuoCNk/671jHmwu0cqk2u/Egrp776zxkEO1xivH
d0EWjJmlDLmQ2hEhkpBq46ji/2m4EWPLqTW/EXf3RzwU8uCEldbxEe2tyZ0d6oBt
NnTE
=AhV7
-END PGP SIGNATURE-


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-05 Thread Flavien Charlon
 It would make more sense to introduce a new script opcode that pushes the
current block height onto the operand stack. Then you could implement
arbitrary logic about which blocks the transaction can be valid in. This
would require that the client revalidate all transactions in its mempool
(really, only those making use of this opcode) whenever the chain tip
changes.

I have to say I like this idea, this would allow someone to prove they
can't spend funds before a given date, and vice versa, prove that the funds
can't ever be spent after a given date (and this is provably prunable,
isn't it?). Of course, there are some risks associated with that, but
nobody is forced to use it.

 flooding the network with unrelated high fee transactions
 in order to push a transaction out to where it can never be mined at
 all.

This becomes increasingly expensive as the deadline is further away, so not
very hard to mitigate.


On Sat, Aug 2, 2014 at 1:36 AM, Tom Harding t...@thinlink.com wrote:

 On 7/31/2014 5:58 PM, Kaz Wesley wrote:
  1. start setting nLockTime to the current height by default in newly
  created transactions (or slightly below the current height, for
  reorg-friendliness)

 Reorg-frendliness is the opposite of the rationale behind #2340, which
 proposes setting nLockTime at current-height + 1 to prevent
 fee-sniping reorgs...


  2. once users have had some time to upgrade to clients that set
  nLockTime, start discouraging transactions without nLockTime --
  possibly with a slightly higher fee required for relay
  3. start rate-limiting relay of transactions without an nLockTime
  (maybe this alone could be used to achieve [2])
  4. add a new IsStandard rule rejecting transactions with an nLockTime
  more than N blocks behind the current tip (for some fixed value N, to
  be determined)
 

 One way to proceed is implement #3753 (mempool janitor) in such a way
 that transactions with nLockTime are allowed to live a bit longer in the
 mempool (say 500 blocks) than those without (72 hours).  In other words,
 as a first step, just actually start expiring things from the mempool in
 bitcoin core, and leave any relay fee adjustments or rate limiting for
 later.  The isStandard change would be a good complement to #3753, to
 avoid relaying a tx that will soon expire by the nLockTime rule anyway.




 --
 Want fast and easy access to all the code in your enterprise? Index and
 search up to 200,000 lines of code with a free copy of Black Duck
 Code Sight - the same software that powers the world's largest code
 search on Ohloh, the Black Duck Open Hub! Try it now.
 http://p.sf.net/sfu/bds
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development

--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-05 Thread Jeff Garzik
Glad this was brought up.

Transaction expiration is something that I have wanted to see happen in
bitcoin for a long, long time.  The user experience of unconfirming
transactions setting around in limbo is just horrible.  Bitcoin software by
necessity has gotten better about attaching fees so this sort of behavior
is uncommon, but that does not eliminate the problem.

Of course, we cannot presume that a transaction will truly disappear -- The
Internet Never Forgets -- but given a bit of mempool adjusting, we can
achieve the next best thing:  the majority of the network forgets the
transaction and becomes willing to relay a respend of some or all of the
inputs.  This uses existing client logic where the client must rebroadcast
a transaction until it is confirmed.

In general, if a transaction has not made it into a block within 144*X
blocks, there is _some_ reason it is getting rejected by the miners.

The mempool janitor is a garbage collector design.  This is inferior to the
superblock model described at
https://github.com/bitcoin/bitcoin/issues/3723   Other models can also
achieve similar results.

There are a lot of issues tied together here:  transaction expiration, the
desire to cap the mempool ram usage, scalability, DoS prevention, ...
mempool ties a lot together.

-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.  https://bitpay.com/
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-05 Thread Alex Mizrahi

 A distinction there is that they can only become invalid via a
 conflict— replaced by another transaction authored by the prior
 signers. If no other transaction could be created (e.g. you're a
 multisigner and won't sign it again) then there is no such risk.


You need to check transaction's dependencies up to a certain depth to know
whether it is safe:
 If one of inputs depends on transaction which is signed by parties with
unknown trustworthiness, then it isn't safe.


  It now introduces chance events (act of god) into the mix where they
 they didn't exist before.


You need to check transaction's dependencies up to a certain depth to know
whether it is safe:
  If one of inputs depends on transaction time-locked script (or other
unrecognized script), then it isn't safe.

Situation is identical, you might need several extra lines of code.

I think it would matter only if we had deterministic, reliable mempool and
reorganization behavior. But it's not something we can depend on.
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-05 Thread Mike Hearn

 The user experience of unconfirming transactions setting around in limbo
 is just horrible.  Bitcoin software by necessity has gotten better about
 attaching fees so this sort of behavior is uncommon, but that does not
 eliminate the problem.


Yes, indeed. I suspect there's a quick hack that could make this problem a
lot better though.

I think I brought up this idea before, but can't quite remember. Anyway I'm
willing to bet that if we analysed the data some more, we'd discover that
most legitimate i.e. non-DoS unconfirmed transactions that sit around for
ages are linked back to the block chain within two hops and not more. That
is people send a transaction that uses up their coin age, and then
immediately those coins are immediately respent again, but then those final
new coins are not spent.

On the other hand DoS attacks look like bouncing your coins around over and
over forever, i.e. more than two or three hops back to the chain.

So I wonder if making priority look back two or three transactions but not
more would help real users a lot, whilst not opening up any significant new
potential for DoS.
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-05 Thread Jeff Garzik
I feel like a lot of this will be driven by implementation, and costs of
changing the implementation.  Additional look-backs are of course doable,
but they incur some disk I/O costs.  The fields available in memory for
each mempool TX are

uint256 tx_hash; // hash of next field
CTransaction tx;
int64_t nFee; // Cached to avoid expensive parent-transaction lookups
size_t nTxSize; // ... and avoid recomputing tx size
int64_t nTime; // Local time when entering the mempool
double dPriority; // Priority when entering the mempool
unsigned int nHeight; // Chain height when entering the mempool

As a first pass, we may prune the mempool without additional db lookups
quite easily based on time criteria.  Or, additional in-memory indexes may
be constructed to maintain hashes ordered by priority/fees.

Those techniques seem likely to be attempted before resorting to looking
back two or three TXs deep at coin age -- which I admit is an interesting
metric.

-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.  https://bitpay.com/
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-05 Thread Kaz Wesley
 In general, if a transaction has not made it into a block within 144*X 
 blocks, there is _some_ reason it is getting rejected by the miners.

This is the crux of my concern: relay policy and miner priorities will
not necessarily always be in sync, and node resource management
shouldn't rely on them being compatible. There are other solutions
than transaction expiration; I think Gavin's idea from the
block-squashing thread, in which miners explicitly provide information
about their policies, would go a long way to address this. But even
when mechanisms for reconciling nodes' expectations about miners'
behavior with miners' actual behavior are available, it may be
desirable to keep an expiry mechanism in place in case of glitches
between node understanding of policy and actual miner policy.

Any approach based on beginning a transaction expiry countdown when a
transaction is received (as in mempool janitor) seems unviable to me:
once a node has forgotten a transaction, it must be susceptible to
reaccepting it; all the functionality of such an expiry mechanism
depends on the network not containing any nodes with slightly
different relay behavior from bitcoind. I could accidentally
debilitate mempool janitors across the entire network if I set up two
nodes to exchange mempools whenever they reconnected to each other,
and restarted each frequently.

That's why I think including information in the transaction itself, as
with my nLockTime/IsStandard proposal, is necessary for transactions
to reliably eventually die off from mempools.
There's a modification I've been thinking about: allow a transaction's
lifetime to be refreshed (even after expiry) by a child transaction,
along the lines of child-pays-for-parent fee policy. This would
eliminate the need to reuse a key to make a replacement for an expired
transaction (when submitting the tx directly to a miner is not an
option), as well as alleviating the potential inconvenience in cases
like Peter brought up, where nLockTime is used for exchanged locked
transactions as part of a multi-transaction contract. With
child-refreshes-parent, a transaction's receivers and senders would
have the ability to keep trying to get their payment confirmed, but
anyone on the network can't just keep all transactions alive.


On Tue, Aug 5, 2014 at 10:48 AM, Jeff Garzik jgar...@bitpay.com wrote:
 Glad this was brought up.

 Transaction expiration is something that I have wanted to see happen in
 bitcoin for a long, long time.  The user experience of unconfirming
 transactions setting around in limbo is just horrible.  Bitcoin software by
 necessity has gotten better about attaching fees so this sort of behavior is
 uncommon, but that does not eliminate the problem.

 Of course, we cannot presume that a transaction will truly disappear -- The
 Internet Never Forgets -- but given a bit of mempool adjusting, we can
 achieve the next best thing:  the majority of the network forgets the
 transaction and becomes willing to relay a respend of some or all of the
 inputs.  This uses existing client logic where the client must rebroadcast a
 transaction until it is confirmed.

 In general, if a transaction has not made it into a block within 144*X
 blocks, there is _some_ reason it is getting rejected by the miners.

 The mempool janitor is a garbage collector design.  This is inferior to the
 superblock model described at
 https://github.com/bitcoin/bitcoin/issues/3723   Other models can also
 achieve similar results.

 There are a lot of issues tied together here:  transaction expiration, the
 desire to cap the mempool ram usage, scalability, DoS prevention, ...
 mempool ties a lot together.

 --
 Jeff Garzik
 Bitcoin core developer and open source evangelist
 BitPay, Inc.  https://bitpay.com/

--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-05 Thread Jeff Garzik
On Tue, Aug 5, 2014 at 3:10 PM, Kaz Wesley kezi...@gmail.com wrote:

 Any approach based on beginning a transaction expiry countdown when a
 transaction is received (as in mempool janitor) seems unviable to me:

...

 That's why I think including information in the transaction itself, as
 with my nLockTime/IsStandard proposal, is necessary for transactions
 to reliably eventually die off from mempools.


reliably die off from mempools leads into the land of tightly
synchronizing memory pools across the network which is a problem of...
large scope and much debate.  :)

For the moment, simply capping the mempool's size at each local node is a
much more reachable goal.  Capping, then, implies some culling policy.  In
general, bitcoind Tx mempool size is rather open ended, and that needs
sorting out.

-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.  https://bitpay.com/
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-05 Thread Tom Harding
On 8/5/2014 12:10 PM, Kaz Wesley wrote:
 Any approach based on beginning a transaction expiry countdown when a 
 transaction is received (as in mempool janitor) seems unviable to me: 
 once a node has forgotten a transaction, it must be susceptible to 
 reaccepting it;

It's hard to argue with that logic.

If nLockTime is used for expiration, transaction creator can't lie to 
help tx live longer without pushing initial confirmation eligibility 
into the future.  Very pretty.  It would also enable fill or kill 
transactions with a backdated nLockTime, which must be confirmed in a 
few blocks, or start vanishing from mempools.


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-08-01 Thread Tom Harding
On 7/31/2014 5:58 PM, Kaz Wesley wrote:
 1. start setting nLockTime to the current height by default in newly
 created transactions (or slightly below the current height, for
 reorg-friendliness)

Reorg-frendliness is the opposite of the rationale behind #2340, which 
proposes setting nLockTime at current-height + 1 to prevent 
fee-sniping reorgs...


 2. once users have had some time to upgrade to clients that set
 nLockTime, start discouraging transactions without nLockTime --
 possibly with a slightly higher fee required for relay
 3. start rate-limiting relay of transactions without an nLockTime
 (maybe this alone could be used to achieve [2])
 4. add a new IsStandard rule rejecting transactions with an nLockTime
 more than N blocks behind the current tip (for some fixed value N, to
 be determined)


One way to proceed is implement #3753 (mempool janitor) in such a way 
that transactions with nLockTime are allowed to live a bit longer in the 
mempool (say 500 blocks) than those without (72 hours).  In other words, 
as a first step, just actually start expiring things from the mempool in 
bitcoin core, and leave any relay fee adjustments or rate limiting for 
later.  The isStandard change would be a good complement to #3753, to 
avoid relaying a tx that will soon expire by the nLockTime rule anyway.



--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-07-31 Thread Peter Todd
On Thu, Jul 31, 2014 at 05:58:23PM -0700, Kaz Wesley wrote:
 I have a proposal for a way to add finite and predictable lifespans to
 transactions in mempools: we d̶e̶s̶t̶r̶o̶y̶ ̶t̶h̶e̶
 ̶r̶e̶s̶u̶r̶r̶e̶c̶t̶i̶o̶n̶ ̶h̶u̶b̶ use nLockTime and a new standardness
 rule. It could be done in stages, would not necessarily require even a
 soft fork, and does not cause problems with reorgs like the proposal
 in #3509:

Anything that changes the semantics of nLockTime will do harm to
existing and future applications that make use of nLockTime for things
like refund transactions.

In any case, why do transactions need finite lifespans in mempools? If
you want to double-spend them with higher fees, then implement
replace-by-fee. In any case, lifetimes will never be deterministic as
not everyone runs the same software.

 Transactions would stop being relayed and drop out of mempools a fixed
 number of blocks from their creation; once that window had passed, the
 sender's wallet could begin to expect the transaction would not be
 confirmed. In case a reorg displaces a transaction until after its
 expiry height, a miner can still put it back in the blockchain; the
 expiry height is just a relay rule. Also, a user who needed to get
 their original expired transaction confirmed could still do so by
 submitting it directly to a miner with suitable policies.

...in which case someone will circumvent this IsStandard() rule by
submitting expired transactions directly to miners with suitable
policies.



signature.asc
Description: Digital signature
--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-07-31 Thread Kaz Wesley
On Thu, Jul 31, 2014 at 6:06 PM, Peter Todd p...@petertodd.org wrote:
 Anything that changes the semantics of nLockTime will do harm to
 existing and future applications that make use of nLockTime for things
 like refund transactions.

I think this would be compatible with most uses of nLockTime -- e.g.,
at the time a refund transaction becomes broadcastable, its
beneficiary would usually have no reason to wait for a long time
before broadcasting it; if they did so (probably because they weren't
online to redeem the refund), they'd need to use the
submit-directly-to-miner option, but they wouldn't lose their refund.

 In any case, why do transactions need finite lifespans in mempools? If
 you want to double-spend them with higher fees, then implement
 replace-by-fee.

Perpetuating transactions that have been in mempools for a long time
and are not being confirmed has been cited as a reason for nodes not
to exchange mempools (#3721, #1833, #3722); it's been implied that it
would be desirable for wallets to wait until a transaction had had a
chance to be accepted before double-spending with a higher fee
(#3722); and an unconfirmed transaction-age-based policy for
preventing mempool accumulation has been advocated (#3753, #3722) [I
hope my summarization is not misrepresenting anyone's opinions here;
please see the arguments made in the actual comments on the bugs].
These discussions are mostly fairly old, but I don't know of any
changes that have been made that provide alternative answers to these
concerns mentioned by at least three different developers.

 In any case, lifetimes will never be deterministic as not everyone runs
 the same software.

That's true, but none of the benefits of these changes require the
policy to be unanimous; most of the effect could be provided by most
of the network following these rules.

 Transactions would stop being relayed and drop out of mempools a fixed
 number of blocks from their creation; once that window had passed, the
 sender's wallet could begin to expect the transaction would not be
 confirmed. In case a reorg displaces a transaction until after its
 expiry height, a miner can still put it back in the blockchain; the
 expiry height is just a relay rule. Also, a user who needed to get
 their original expired transaction confirmed could still do so by
 submitting it directly to a miner with suitable policies.

 ...in which case someone will circumvent this IsStandard() rule by
 submitting expired transactions directly to miners with suitable
 policies.

Yes, that is a feature. None of the benefits of transaction expiration
rely on expiration being final, and any possible downsides of
expiration are largely mitigated by the option still being available
to get expired transactions mined.

--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-07-31 Thread Matt Whitlock
It would make more sense to introduce a new script opcode that pushes the 
current block height onto the operand stack. Then you could implement arbitrary 
logic about which blocks the transaction can be valid in. This would require 
that the client revalidate all transactions in its mempool (really, only those 
making use of this opcode) whenever the chain tip changes.


On Thursday, 31 July 2014, at 5:58 pm, Kaz Wesley wrote:
 There is currently little in place for managing transaction lifetime
 in the network's mempools (see discussion in github in #3722 mempool
 transaction expiration, and it seems to be a major factor blocking
 some mempool exchange, see #1833/1918, #3721). Expiry per-node a
 certain amount of wall time after receipt has been proposed, but
 that's a fragile mechanism -- a single node could keep all relayable
 transactions alive forever by remembering transactions until most
 nodes have dropped them and then releasing them back into the wild.
 
 I have a proposal for a way to add finite and predictable lifespans to
 transactions in mempools: we d̶e̶s̶t̶r̶o̶y̶ ̶t̶h̶e̶
 ̶r̶e̶s̶u̶r̶r̶e̶c̶t̶i̶o̶n̶ ̶h̶u̶b̶ use nLockTime and a new standardness
 rule. It could be done in stages, would not necessarily require even a
 soft fork, and does not cause problems with reorgs like the proposal
 in #3509:
 1. start setting nLockTime to the current height by default in newly
 created transactions (or slightly below the current height, for
 reorg-friendliness)
 2. once users have had some time to upgrade to clients that set
 nLockTime, start discouraging transactions without nLockTime --
 possibly with a slightly higher fee required for relay
 3. start rate-limiting relay of transactions without an nLockTime
 (maybe this alone could be used to achieve [2])
 4. add a new IsStandard rule rejecting transactions with an nLockTime
 more than N blocks behind the current tip (for some fixed value N, to
 be determined)
 
 Transactions would stop being relayed and drop out of mempools a fixed
 number of blocks from their creation; once that window had passed, the
 sender's wallet could begin to expect the transaction would not be
 confirmed. In case a reorg displaces a transaction until after its
 expiry height, a miner can still put it back in the blockchain; the
 expiry height is just a relay rule. Also, a user who needed to get
 their original expired transaction confirmed could still do so by
 submitting it directly to a miner with suitable policies.

--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-07-31 Thread Gregory Maxwell
On Thu, Jul 31, 2014 at 6:38 PM, Matt Whitlock b...@mattwhitlock.name wrote:
 It would make more sense to introduce a new script opcode that pushes the 
 current block height onto the operand stack. Then you could implement 
 arbitrary logic about which blocks the transaction can be valid in. This 
 would require that the client revalidate all transactions in its mempool 
 (really, only those making use of this opcode) whenever the chain tip changes.

Transactions that become invalid later are have pretty severe
consequences because they might mean that completely in an absence of
fraud transactions are forever precluded due to a otherwise harmless
reorg.

While there may be uses for that, the resulting outputs should be
considered differently fungible— like coinbases which are immature—
and should probably be only used with great caution... not as a
mechanism for ordinary transactions.

--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-07-31 Thread Matt Whitlock
On Thursday, 31 July 2014, at 7:28 pm, Gregory Maxwell wrote:
 On Thu, Jul 31, 2014 at 6:38 PM, Matt Whitlock b...@mattwhitlock.name wrote:
  It would make more sense to introduce a new script opcode that pushes the 
  current block height onto the operand stack. Then you could implement 
  arbitrary logic about which blocks the transaction can be valid in. This 
  would require that the client revalidate all transactions in its mempool 
  (really, only those making use of this opcode) whenever the chain tip 
  changes.
 
 Transactions that become invalid later are have pretty severe
 consequences because they might mean that completely in an absence of
 fraud transactions are forever precluded due to a otherwise harmless
 reorg.

I understand what you're saying, but I don't understand why it's a problem. 
Transactions shouldn't be considered final until a reasonable number of 
confirmations anyway, so the possibility that an accepted transaction could 
become invalid due to a chain reorganization is not a new danger. Ordinary 
transactions can similarly become invalid due to chain reorganizations, due to 
inputs already having been spent in the new branch.

--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] deterministic transaction expiration

2014-07-31 Thread Gregory Maxwell
On Thu, Jul 31, 2014 at 8:26 PM, Matt Whitlock b...@mattwhitlock.name wrote:
 I understand what you're saying, but I don't understand why it's a problem. 
 Transactions shouldn't be considered final until a reasonable number of 
 confirmations anyway, so the possibility that an accepted transaction could 
 become invalid due to a chain reorganization is not a new danger. Ordinary 
 transactions can similarly become invalid due to chain reorganizations, due 
 to inputs already having been spent in the new branch.

A distinction there is that they can only become invalid via a
conflict— replaced by another transaction authored by the prior
signers. If no other transaction could be created (e.g. you're a
multisigner and won't sign it again) then there is no such risk.  It
now introduces chance events (act of god) into the mix where they
they didn't exist before.  Basically it takes was what is a very loose
one way coupling and makes it much tighter. I'm sure if you spend a
bit thinking you can come up with some more corner cases that it would
expose— e.g. flooding the network with unrelated high fee transactions
in order to push a transaction out to where it can never be mined at
all.

--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development