[Bitcoin-development] First-Seen-Safe Replace-by-Fee patch against Bitcoin Core v0.10.2

2015-06-10 Thread Peter Todd
First-seen-safe Replace-by-Fee is now available as a patch against
v0.10.2:

https://github.com/petertodd/bitcoin/tree/first-seen-safe-rbf-v0.10.2

I've also had a pull-req against git HEAD open for a few weeks now:

https://github.com/bitcoin/bitcoin/pull/6176#issuecomment-104877829

I've got some hashing power interested in running this patch in the near
future, so I'm offering a bounty of up to 1 BTC to anyone who can find a
way to attack miners running this patch. Specifically, I'm concerned
about things that would lead to significant losses for those miners. A
total crash would be considered very serious - 1 BTC - while excess
bandwidth usage would be considered minor - more like 0.1 BTC. (remember
that this would have to be bandwidth significantly in excess of existing
attacks)

For reference, here's an example of a crash exploit found by Suhas
Daftuar: https://github.com/bitcoin/bitcoin/pull/6176#issuecomment-104877829

If two people report the same or overlapping issues, first person will
get priority. Adding a new test that demos your exploit to the unit
tests will be looked upon favorably. That said, in general I'm not going
to make any hard promises with regards to payouts and will be using my
best judgement. I've got a bit over 2BTC budgetted for this, which is
coming out of my own pockets - I'm not rich! All applicants are however
welcome to troll me on reddit if you think I'm being unfair.


Suhas: speaking of, feel free to email me a Bitcoin address! :)

-- 
'peter'[:-1]@petertodd.org
06dd456cf5ff8bbb56cf88e9314711d55b75c8d23cccddd5


signature.asc
Description: Digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Is SourceForge still trustworthy enough to host this list?

2015-06-10 Thread Wladimir J. van der Laan
On Wed, Jun 10, 2015 at 10:25:12AM +0200, xor wrote:
 http://www.howtogeek.com/218764/warning-don%E2%80%99t-download-software-from-sourceforge-if-you-can-help-it/

All our downloads (even old ones) have recently been deleted from sourceforge, 
for this reason. They haven't been mentioned in Bitcon Core release 
announcements for a long time.

No opinion on the mailing list. Though I think it's less urgent. The issue of 
moving the mailinglist has come up before a few times and people can't agree 
where to move to.

Wladimir


--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


[Bitcoin-development] Is SourceForge still trustworthy enough to host this list?

2015-06-10 Thread xor
http://www.howtogeek.com/218764/warning-don%E2%80%99t-download-software-from-sourceforge-if-you-can-help-it/

TL;DR:

 In 2013, GIMP’s developers pulled the GIMP Windows downloads from
 SourceForge. SourceForge was full of misleading advertisements
 masquerading as “Download” buttons — something that’s a problem all over
 the web. 
[...]
 In 2015, SourceForge pushed back. Considering the old GIMP account on
 SourceForge “abandoned,” they took control over it, locking out the
 original maintainer. They then put GIMP downloads back up on SourceForge,
 wrapped in SourceForge’s own junkware-filled installer.

signature.asc
Description: This is a digitally signed message part.
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Is SourceForge still trustworthy enough to host this list?

2015-06-10 Thread Andy Schroder


Andy Schroder

On 06/10/2015 03:20 PM, Peter Todd wrote:

On Wed, Jun 10, 2015 at 03:12:02PM -0400, Andy Schroder wrote:

Andy Schroder

On 06/10/2015 03:03 PM, Peter Todd wrote:

4. Seems like digital signatures are always broken on messages because
the list server slightly modifies them (?), so my e-mail client
doesn't verify them all.

What type of digital signatures specifically? What email client?

I think they are usually PGP/MIME signatures that are not working
right. If you'll notice from my e-mail headers:

User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 
Thunderbird/24.2.0
X-Enigmail-Version: 1.6

It might be that Thunderbird doesn't properly handle messages with both
signed and unsigned content. I use mutt myself, which handles it just
fine. (the sigs on your emails verify just fine for instance)



It's possible that the enigmail extension is not working right, but I 
was under the impression that it is just feeding data to gpg and then 
receiving the response back. It's possible that your e-mail you just 
checked was not sent through mailman since I also replied directly to 
you explicitly (in which case the message has not been modified) and you 
probably have the setting in the mailing list set to not send duplicate 
messages if you are an explicit TO. I just deleted all explicit TOs for 
this message, so everyone should be receiving it through the mailing 
list and not directly. Is the signature still valid for you now? I think 
enigmail can handle messages with some signed and unsigned content, and 
maybe PGP/MIME inherently does not support this and a mailing list 
re-writing parts of messages is an expected action? If this message 
re-writing is an expected action and I'm correct that PGP/MIME does not 
support partially signed content, then maybe it is just a recommendation 
for this mailing list to not use PGP/MIME for messages sent to the list?


Can anyone else confirm?





signature.asc
Description: OpenPGP digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Is SourceForge still trustworthy enough to host this list?

2015-06-10 Thread Jeff Garzik
On Wed, Jun 10, 2015 at 11:59 AM, Andy Schroder i...@andyschroder.com
wrote:

  Hello,

 A couple of motivations for a mailing list switch:

1. Sometimes the mailing list delays delivery for 10 minutes to
several days.
2. There are usually lots of ads at the footer of the messages. Really
confuses new readers (for me at least), and seems like it really pollutes
such a historical dialog that may be referenced long into the future. How
would it be if the 10 Commandments, Magna Carta, Bill of Rights, The Sermon
on the Mount, or The Gettysburg Address had ads intertwined within them?
 3. Don't think HTML messages are allowed.
4. Seems like digital signatures are always broken on messages because
the list server slightly modifies them (?), so my e-mail client doesn't
verify them all.

 Not only -- mail header rewrites cause all my emails to go into people's
spam folders, if they were not directly listed in the To/CC headers...





1.



 Andy Schroder

 On 06/10/2015 02:36 PM, s7r wrote:

 The mail list is public, so it's not like the data on it is somehow
 sensitive. Sourcefoge is fine, it has a nice web UI where you can browse
 the message and sort/order them as you want, etc.

 Why would you want to move to a paid solution? And why would you want
 users to have to pay per message? This is the worst idea ever from my
 point of view. We want to encourage people to join the community, run
 full nodes, ask questions, come with solutions, ideas for improvements
 and so on. Everyone should read and write and contribute as much as
 possible with ideas in debates. You never know who can have bright ideas
 in some contexts.

 Bottom line is so far sourceforge handles the mail lists just fine. I
 don't see a single advantage another mail list provider / system could
 offer, except some headache and extra work for migration. The software
 distribution via sourcefoge was cancelled for obvious reasons which I
 fully understand and agree to, but it has nothing to do with the mail
 lists. We have way more important things to brainstorm about.

 On 6/10/2015 7:46 PM, Andy Schroder wrote:

  Regarding changing the e-mail list provider. Is anyone interested in
 sponsoring it? There are non-free options, but it may be difficult to
 always ensure the fee is being paid to the provider. I think finding an
 agreeable free solution may have been the issue before? I've also
 thought of trying to make a pay per message or byte solution (and this
 cost could be dynamic based upon the number of current mailing list
 subscribers). This could solve the who pays problem (the sender pays),
 as well as motivate people to be more concise and clear with their
 messages, and at the same time limit spam.



 Any thoughts?

 Andy Schroder

 On 06/10/2015 05:35 AM, Wladimir J. van der Laan wrote:

  On Wed, Jun 10, 2015 at 10:25:12AM +0200, xor wrote:

  
 http://www.howtogeek.com/218764/warning-don%E2%80%99t-download-software-from-sourceforge-if-you-can-help-it/

  All our downloads (even old ones) have recently been deleted from 
 sourceforge, for this reason. They haven't been mentioned in Bitcon Core 
 release announcements for a long time.

 No opinion on the mailing list. Though I think it's less urgent. The issue of 
 moving the mailinglist has come up before a few times and people can't agree 
 where to move to.

 Wladimir


 --

  
 --
 ___
 Bitcoin-development mailing 
 listBitcoin-development@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bitcoin-development




 --

 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development




-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.  https://bitpay.com/
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism

2015-06-10 Thread Peter Todd
On Wed, Jun 10, 2015 at 02:00:27PM -0600, Nathan Wilcox wrote:
 On Wed, Jun 10, 2015 at 1:19 PM, Aaron Voisine vois...@gmail.com wrote:
 
  It could be done by agreeing on a data format and encoding it in an
  op_return output in the coinbase transaction. If it catches on it could
  later be enforced with a soft fork.
 
 
 Sounds plausible, except SPV protocols would need to include this coinbase
 txn if it's going to help SPV clients. (Until a softfork is activated, SPV
 clients should not rely on this encoding, since until that time the results
 can be fabricated by individual miners.)

Fee stats can always be fabricated by individual miners because fees can
be paid out-of-band.

-- 
'peter'[:-1]@petertodd.org
1245bd2f5c99379ee76836227ded9c08324894faabc0d27f


signature.asc
Description: Digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism

2015-06-10 Thread Nathan Wilcox
On Wed, Jun 10, 2015 at 1:19 PM, Aaron Voisine vois...@gmail.com wrote:

 It could be done by agreeing on a data format and encoding it in an
 op_return output in the coinbase transaction. If it catches on it could
 later be enforced with a soft fork.


Sounds plausible, except SPV protocols would need to include this coinbase
txn if it's going to help SPV clients. (Until a softfork is activated, SPV
clients should not rely on this encoding, since until that time the results
can be fabricated by individual miners.)


 For real up-to-the-minute fee calculations you're also going to want to
 look at the current mempool, how many transactions are waiting, what fees
 they're paying, etc, but of course that information is susceptible to sybil
 attack.


Hm, when you mention Sybil attack, I don't quite follow.

When a client relies on any report of a mempool [*], this is already
outside the realm of locally-verifiable SPV information, so they are
already susceptible to the service making false claims. If that's
acceptable (and in many cases it may be) then this whole mechanism is moot,
because the client can ask the service for fee statistics for past blocks.


 In practice what we're doing for now is using services like blockcypher
 who's business is improving reliability of zero-conf to tell us what
 fee-per-kb is needed, and then putting a hard coded range around it to
 protect against the service being compromised.


This is interesting for me, because I had previously believed fees were
fairly static presently, and also because I like hearing about real life
wallet implementations.

So if this SPV Fee Stats feature were added, a wallet might rely on an
API for timely stats (aka block height  1) then verify that the API
isn't lying after doing SPV verification of fee stats for confirmed blocks.


This is also the kind of thing being done for exchange rate data which is
 probably the bigger security risk until bitcoin becomes the standard unit
 of account for the planet.


That makes sense, although there's no SPV equivalent for exchange data.


Aaron Voisine
 co-founder and CEO
 breadwallet.com

 On Wed, Jun 10, 2015 at 10:37 AM, Nathan Wilcox nat...@leastauthority.com
  wrote:

 [I'm currently wading through bitcoin-development. I'm still about a
 month behind, so I apologize in advance for any noisy redundancy in this
 post.]

 While reading about blocksize, I've just finished Mike Hearn's blog post
 describing expected systemic behavior as actual blocks approach the current
 limit (with or without non-protocol-changing implementation improvements):

 https://medium.com/@octskyward/crash-landing-f5cc19908e32


 One detail Mike uses to argue against the fee's will save us line of
 reasoning is that wallets have no good way to learn fee information.

 So, here's a proposal to fix that: put fee and (and perhaps block size,
 UTXO, etc...) statistics into the locally-verifiable data available to SPV
 clients (ie: block headers).


 It's easy to imagine a hard fork that places details like per-block total
 fees, transaction count, fee variance, UTXO delta, etc... in a each block
 header. This would allow SPV clients to rely on this data with the same
 PoW-backed assurances as all other header data.

 This mechanism seems valuable regardless of the outcome of blocksize
 debate. So long as fees are interesting or important, SPV clients should
 know about them. (Same for other stats such as UTXO count.)

 Upgrading the protocol without a hard-fork may be possible and is left as
 an exercise for the reader. ;-)

 --
 Nathan Wilcox
 Least Authoritarian

 email: nat...@leastauthority.com
 twitter: @least_nathan
 PGP: 11169993 / AAAC 5675 E3F7 514C 67ED  E9C9 3BFE 5263 1116 9993


 --

 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development





-- 
Nathan Wilcox
Least Authoritarian

email: nat...@leastauthority.com
twitter: @least_nathan
PGP: 11169993 / AAAC 5675 E3F7 514C 67ED  E9C9 3BFE 5263 1116 9993
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Is SourceForge still trustworthy enough to host this list?

2015-06-10 Thread Andy Schroder

Hello,

Thanks for testing this clarifying things about PGP/MIME and I apologize 
for wasting your time with it. It looks like a SPAM filtering service I 
use is re-writing some parts of some plain text messages with some 
special/alternate encoding characters (not sure what it really is). 
Anyway, if I manually export/import a message from gmane (bypassing my 
e-mail SPAM filter), thunderbird/enigmail is not having problems 
verifying signatures. I guess I never realized this before because all 
other signed messages I normally receive are encrypted and the SPAM 
filter does not mess with non plain text data.




Andy Schroder

On 06/10/2015 03:43 PM, Peter Todd wrote:

On Wed, Jun 10, 2015 at 03:36:42PM -0400, Andy Schroder wrote:

It's possible that the enigmail extension is not working right, but
I was under the impression that it is just feeding data to gpg and
then receiving the response back. It's possible that your e-mail you
just checked was not sent through mailman since I also replied
directly to you explicitly (in which case the message has not been
modified) and you probably have the setting in the mailing list set
to not send duplicate messages if you are an explicit TO. I just
deleted all explicit TOs for this message, so everyone should be
receiving it through the mailing list and not directly. Is the
signature still valid for you now? I think enigmail can handle

It has perfectly valid signatures, as do your earlier messages to the
list.


messages with some signed and unsigned content, and maybe PGP/MIME
inherently does not support this and a mailing list re-writing parts
of messages is an expected action? If this message re-writing is an
expected action and I'm correct that PGP/MIME does not support
partially signed content, then maybe it is just a recommendation for
this mailing list to not use PGP/MIME for messages sent to the list?

PGP/MIME definitely does support partially signed content.






signature.asc
Description: OpenPGP digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Is SourceForge still trustworthy enough to host this list?

2015-06-10 Thread s7r
The mail list is public, so it's not like the data on it is somehow
sensitive. Sourcefoge is fine, it has a nice web UI where you can browse
the message and sort/order them as you want, etc.

Why would you want to move to a paid solution? And why would you want
users to have to pay per message? This is the worst idea ever from my
point of view. We want to encourage people to join the community, run
full nodes, ask questions, come with solutions, ideas for improvements
and so on. Everyone should read and write and contribute as much as
possible with ideas in debates. You never know who can have bright ideas
in some contexts.

Bottom line is so far sourceforge handles the mail lists just fine. I
don't see a single advantage another mail list provider / system could
offer, except some headache and extra work for migration. The software
distribution via sourcefoge was cancelled for obvious reasons which I
fully understand and agree to, but it has nothing to do with the mail
lists. We have way more important things to brainstorm about.

On 6/10/2015 7:46 PM, Andy Schroder wrote:
 Regarding changing the e-mail list provider. Is anyone interested in 
 sponsoring it? There are non-free options, but it may be difficult to 
 always ensure the fee is being paid to the provider. I think finding an 
 agreeable free solution may have been the issue before? I've also 
 thought of trying to make a pay per message or byte solution (and this 
 cost could be dynamic based upon the number of current mailing list 
 subscribers). This could solve the who pays problem (the sender pays), 
 as well as motivate people to be more concise and clear with their 
 messages, and at the same time limit spam.
 
 
 
 Any thoughts?
 
 Andy Schroder
 
 On 06/10/2015 05:35 AM, Wladimir J. van der Laan wrote:
 On Wed, Jun 10, 2015 at 10:25:12AM +0200, xor wrote:
 http://www.howtogeek.com/218764/warning-don%E2%80%99t-download-software-from-sourceforge-if-you-can-help-it/
 All our downloads (even old ones) have recently been deleted from 
 sourceforge, for this reason. They haven't been mentioned in Bitcon Core 
 release announcements for a long time.

 No opinion on the mailing list. Though I think it's less urgent. The issue 
 of moving the mailinglist has come up before a few times and people can't 
 agree where to move to.

 Wladimir


 --
 
 
 --
 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development
 

--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism

2015-06-10 Thread Aaron Voisine
The other complication is that this will tend to be a lagging indicator
based on network congestion from the last time you connected. If we assume
that transactions are being dropped in an unpredictable way when blocks are
full, knowing the network congestion *right now* is critical, and even then
you just have to hope that someone who wants that space more than you do
doesn't show up after you disconnect.


Aaron Voisine
co-founder and CEO
breadwallet.com

On Wed, Jun 10, 2015 at 1:26 PM, Mike Hearn m...@plan99.net wrote:

 I described an alternative way for SPV wallets to learn about fees some
 time ago. It requires a new transaction version that embeds output values
 into the signed data. Then an upgrade to the P2P protocol to send UTXO data
 along with transactions when they are relayed.

 The idea is that the wallet sets a Bloom filter with an FP rate that
 ensures it will see some random subset of all transactions being broadcast
 on the network, and with the extra data, it can calculate the fee paid.
 Once a transaction broadcast is observed the wallet includes that tx hash
 in its next Bloom filter, thus it can see which block the tx confirmed in.
 By measuring the amount of time that passed between a broadcast and it
 appearing in a block, it can calculate its own tables of fee paid:time
 taken.

 This has the advantage that you don't have to trust miners to publish data
 accurately. However it requires some protocol upgrades and of course, a lot
 of new code in SPV wallets.

 The way Bitcoin Wallet for Android handles fees currently is to just
 update a hard coded value every so often.


 --

 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development


--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


Re: [Bitcoin-development] Proposal: SPV Fee Discovery mechanism

2015-06-10 Thread Aaron Voisine
 Sounds plausible, except SPV protocols would need to include this
coinbase txn if it's going to help SPV clients.

Yes you'd either need a way to add those transactions to the bloom filter,
or add/modify a p2p message to request it specifically.

 when you mention Sybil attack, I don't quite follow.

I just mean that someone could spin up a bunch of malicious p2p nodes that
lied about mempool data. It's a bit worse for SPV clients since they can't
verify that unconfirmed transactions are valid.

 I had previously believed fees were fairly static presently,

I actually just added it the other day after getting blockcypher to include
it in their api. The current release is still using a hard coded fee rate.

Aaron Voisine
co-founder and CEO
breadwallet.com

On Wed, Jun 10, 2015 at 1:00 PM, Nathan Wilcox nat...@leastauthority.com
wrote:

 On Wed, Jun 10, 2015 at 1:19 PM, Aaron Voisine vois...@gmail.com wrote:

 It could be done by agreeing on a data format and encoding it in an
 op_return output in the coinbase transaction. If it catches on it could
 later be enforced with a soft fork.


 Sounds plausible, except SPV protocols would need to include this coinbase
 txn if it's going to help SPV clients. (Until a softfork is activated, SPV
 clients should not rely on this encoding, since until that time the results
 can be fabricated by individual miners.)


 For real up-to-the-minute fee calculations you're also going to want to
 look at the current mempool, how many transactions are waiting, what fees
 they're paying, etc, but of course that information is susceptible to sybil
 attack.


 Hm, when you mention Sybil attack, I don't quite follow.

 When a client relies on any report of a mempool [*], this is already
 outside the realm of locally-verifiable SPV information, so they are
 already susceptible to the service making false claims. If that's
 acceptable (and in many cases it may be) then this whole mechanism is moot,
 because the client can ask the service for fee statistics for past blocks.


 In practice what we're doing for now is using services like blockcypher
 who's business is improving reliability of zero-conf to tell us what
 fee-per-kb is needed, and then putting a hard coded range around it to
 protect against the service being compromised.


 This is interesting for me, because I had previously believed fees were
 fairly static presently, and also because I like hearing about real life
 wallet implementations.

 So if this SPV Fee Stats feature were added, a wallet might rely on an
 API for timely stats (aka block height  1) then verify that the API
 isn't lying after doing SPV verification of fee stats for confirmed blocks.


 This is also the kind of thing being done for exchange rate data which is
 probably the bigger security risk until bitcoin becomes the standard unit
 of account for the planet.


 That makes sense, although there's no SPV equivalent for exchange data.


 Aaron Voisine
 co-founder and CEO
 breadwallet.com

 On Wed, Jun 10, 2015 at 10:37 AM, Nathan Wilcox 
 nat...@leastauthority.com wrote:

 [I'm currently wading through bitcoin-development. I'm still about a
 month behind, so I apologize in advance for any noisy redundancy in this
 post.]

 While reading about blocksize, I've just finished Mike Hearn's blog post
 describing expected systemic behavior as actual blocks approach the current
 limit (with or without non-protocol-changing implementation improvements):

 https://medium.com/@octskyward/crash-landing-f5cc19908e32


 One detail Mike uses to argue against the fee's will save us line of
 reasoning is that wallets have no good way to learn fee information.

 So, here's a proposal to fix that: put fee and (and perhaps block size,
 UTXO, etc...) statistics into the locally-verifiable data available to SPV
 clients (ie: block headers).


 It's easy to imagine a hard fork that places details like per-block
 total fees, transaction count, fee variance, UTXO delta, etc... in a each
 block header. This would allow SPV clients to rely on this data with the
 same PoW-backed assurances as all other header data.

 This mechanism seems valuable regardless of the outcome of blocksize
 debate. So long as fees are interesting or important, SPV clients should
 know about them. (Same for other stats such as UTXO count.)

 Upgrading the protocol without a hard-fork may be possible and is left
 as an exercise for the reader. ;-)

 --
 Nathan Wilcox
 Least Authoritarian

 email: nat...@leastauthority.com
 twitter: @least_nathan
 PGP: 11169993 / AAAC 5675 E3F7 514C 67ED  E9C9 3BFE 5263 1116 9993


 --

 ___
 Bitcoin-development mailing list
 Bitcoin-development@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bitcoin-development





 --
 Nathan Wilcox
 Least Authoritarian

 email: nat...@leastauthority.com
 twitter: @least_nathan
 PGP: 11169993 / 

Re: [Bitcoin-development] Lexicographical Indexing of Transaction Inputs and Outputs

2015-06-10 Thread Kristov Atlas
Thanks for the feedback. I think I have reflected all of your requested
changes in the latest version, in the BIP and sample code:

https://github.com/kristovatlas/rfc/tree/master/bips

-Kr

On Tue, Jun 9, 2015 at 4:14 PM, Peter Todd p...@petertodd.org wrote:

 On Mon, Jun 08, 2015 at 06:53:54PM -0400, Kristov Atlas wrote:

 Two other things:



  On Sat, Jun 6, 2015 at 10:35 PM, Peter Todd p...@petertodd.org wrote:
 
   Why mention SIGHASH_SINGLE at all? Its use-case is highly specialized
   protocols; you haven't taken into account the needs of those protocols.
   For BIP's it's better to stick to the use-cases where the need is clear
   and there exists running code that to speculate too much on future
 uses.
   With signature hashing in particular it's not yet clear at all what
   future OP_CHECKSIG's will look like, let alone the various ways people
   will use sighash for smart contract type stuff.
  
   You'd be better off presenting the BIP in terms of a generic statement
   that except when otherwise prevented by advanced signature hashing
   requirements, wallet software must emit transactions sorted according
 to
   the following You can then specify the two common cases in detail:
  
   1) SIGHASH_ALL: input and output order signed, so sort appropriately
  
   2) SIGHASH_ANYONECANPAY: input order not signed, so software should
 emit
  transactions sorted, recognising that the actual mined order may be
  changed.
  
 
  That makes sense. I updated the language as follows -- your thoughts?
 Keep
  in mind this BIP is informational, and so people are free to ignore it.
 
  Applicability: This BIP applies to all transactions of signature hash
 type
  SIGHASH_ALL. Additionally,  software compliant with this BIP that allows
  later parties to update the transaction (e.g. using signature hash types
  SIGHASH_NONE or a variant of SIGHASH_ANYONECANPAY) should emit
  lexicographically sorted inputs and outputs, although they may later be
  modified. Transactions that have index dependencies between transactions
 or
  within the same transaction are covered under the section of this BIP
  entitled “Handling Input/Output Dependencies.”

 I'd keep it even simpler than that, and just say for now that such
 use-cases are out of the scope of this BIP, however those standards
 should come up with some kind of deterministic standard that meets the
 needs of the protocol. Again, there's a bunch of possible use-cases here
 and we just can't predict them; focus on the fact that the *spirit* of
 what this BIP is about is applicable and future standards should be
 developed.

 So I'd change the Applicability section to:

 This BIP applies to all transactions where the order of inputs and
 outputs does not matter. This is true for the vast majority of
 transactions as they simply move funds from one place to another.

 Currently this generally refers to transactions where SIGHASH_ALL is
 used, in which case the signatures commit to the exact order of input
 and outputs. In the case where SIGHASH_ANYONECANPAY and/or SIGHASH_NONE
 has been used (e.g. crowdfunds) the order of inputs and/or outputs may
 not be signed, however compliant software should still emit transactions
 with sorted inputs and outputs, even though they may later be modified
 by others.

 In the event that future protocol upgrades introduce new signature hash
 types, compliant software should apply the lexographic ordering
 principle analogously.

 While out of scope of this BIP, protocols that do require a specified
 order of inputs/outputs (e.g. due to use of SIGHASH_SINGLE) should
 consider the goals of this BIP and how best to adapt them to the
 specifics needs of those protocols.


 Then remove the handling input/output deps section.

   Do you have a patch implementing deterministic tx ordering for Bitcoin
   Core yet?
  
 
  I'm not a frequent C programmer, so I'd prefer to let someone else take
  care of it, as a frequent committer of code would do a faster and more
  stylistically consistent job of it. If no one else will, however, I will.



 re: the actual ordering algorithm, having txids be sorted by with the
 hex-based algorithm is odd. I'd simply say they're sorted as
 little-endian byte arrays, or in other words, with the bytearr_cmp()
 function, but with the order of bytes reversed. You also should say that
 we're doing that to make the user see them in visually sorted order to
 match expectations because txids are displayed as little-endian.

 For outputs, don't say locking script, say scriptPubKey. Secondly,
 scriptPubKeys are not in little-endian representation - they have no
 endianness to them. With output amount, there's no need to say that
 they're unsigned or little-endian satoshies, just say they're sorted
 largest/smallest amount first.

 For the sake of efficiency, amounts will be considered first for
 sorting, since they contain fewer bytes of information (7 bytes)
 compared to a standard P2PKH locking script (800