Re: [Bitcoin-development] Floating fees and SPV clients
On Sun, Dec 01, 2013 at 07:18:07PM +0100, Mike Hearn wrote: > > Bitcoin is and always will be limited in capacity - transactions may not > > confirm in a reasonable about of time because of high-demand and/or DoS > > attacks. > > I agree in the general case, but I was talking about the mobile wallet case > specifically (i.e. people who are sending money between themselves or making > small purchases of physical things). I think Bitcoin should be able to scale > to handle these sorts of ordinary every-day transactions. Where I’d expect to > see transactions falling off the edge is in more specialised cases like very > small single micropayments, or “optional” internal transactions like > mixing/re/defragmentation of wallets that don’t correspond to an actual > payment. Those sorts of transactions would I guess be the first to go when > faced with a sudden capacity crunch, but they wouldn’t show up in a mobile > wallet UI anyway. Maybe, maybe not. We have no idea what fees will be because the system's entire capacity is, and always will be, limited. That's just how fundementally unscalable systems with huge global state work. What demand will be for that limited capacity is unknown. > > re: merchants paying tx fees, child-pays-for-parent is inefficient > > I know the existing code is, but is that fundamentally the case or just how > the code has been written? I haven’t looked at this issue much but I know > you’ve worked on it, so I’m curious to learn about why it’s inefficient and > whether there are any fixes possible. No, Luke's existing code uses good algorithms with O(n) scaling for n transactions. The inefficiency is needing a second transaction, bloating the blockchain and driving up fees. -- 'peter'[:-1]@petertodd.org 000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3 signature.asc Description: Digital signature -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
> Bitcoin is and always will be limited in capacity - transactions may not > confirm in a reasonable about of time because of high-demand and/or DoS > attacks. I agree in the general case, but I was talking about the mobile wallet case specifically (i.e. people who are sending money between themselves or making small purchases of physical things). I think Bitcoin should be able to scale to handle these sorts of ordinary every-day transactions. Where I’d expect to see transactions falling off the edge is in more specialised cases like very small single micropayments, or “optional” internal transactions like mixing/re/defragmentation of wallets that don’t correspond to an actual payment. Those sorts of transactions would I guess be the first to go when faced with a sudden capacity crunch, but they wouldn’t show up in a mobile wallet UI anyway. > re: merchants paying tx fees, child-pays-for-parent is inefficient I know the existing code is, but is that fundamentally the case or just how the code has been written? I haven’t looked at this issue much but I know you’ve worked on it, so I’m curious to learn about why it’s inefficient and whether there are any fixes possible. smime.p7s Description: S/MIME cryptographic signature -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
On Sun, Dec 01, 2013 at 06:19:14PM +0100, Mike Hearn wrote:
> > Both can be combined into adapting the current generic messages ("This
> > payment should become spendable shortly" for incoming and "This payment
> > has not been transmitted yet" for outgoing transactions).
>
> What would the new messages say?
>
> We need to get away from the notion of senders attaching fees anyway. This is
> the wrong way around because it’s the recipient who cares about double
> spending risk, not the sender. That’s why merchants keep running into issues
> with people attaching zero fees. Of course they attach zero fees. They know
> they aren’t going to double spend. It’s the merchant who cares about getting
> the security against that.
>
> The UI for sending money should end up dead simple - no mention of fees
> anywhere, IMO.
>
> The UI for receiving money could be a bit more complicated but even then - I
> think if ordinary people using smartphone wallets are having to think about
> how quickly they want their transaction to confirm and adjust fees, etc on
> the receiving side then we’re getting dangerously close to the usability
> failure zone.
>
> Unfortunately we lack the protocol pieces to get the right UI here :( Someone
> needs to sit down and figure out what the UI *should* look like, in the ideal
> world, and then work backwards to figure out what needs to be done to get us
> there.
>
> > For outgoing transactions, if it is very clear that they're never going
> > to be confirmed, I'd like to see a "Revoke" button.
>
> Disagree. There should never be any cases in which a transaction doesn’t
> confirm. Period. I know there have been bugs with bitcoinj that could cause
> this in the past, but they were bugs and they got fixed/will get fixed.
>
> Settlement failure is just unacceptable and building a UI around the
> possibility will just encourage people to think of it as normal, when it
> should not be so.
Bitcoin is and always will be limited in capacity - transactions may not
confirm in a reasonable about of time because of high-demand and/or DoS
attacks. Giving senders and/or receivers the ability to increase fees
after the fact is the only way you'll ever be able to deal with these
situations. Of course, in those situations revoke isn't going to be 100%
reliable until the txins get spent elsewhere, but that just indicates
the UI problem is around that kind of functionality is subtle.
re: merchants paying tx fees, child-pays-for-parent is inefficient, and
micropayments direct to miners isn't implemented. (though I did write up
a rough sketch of how to do that in a decentralized fashion on
#bitcoin-dev) Propose something concrete.
--
'peter'[:-1]@petertodd.org
000f9102d27cfd61ea9e8bb324593593ca3ce6ba53153ff251b3
signature.asc
Description: Digital signature
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
> "This payment did not become spendable since xxx minutes. Check with the > sender if s/he paid the Bitcoin network fee. Check if your internet > connection is working properly." (incoming) That seems reasonable. The other message should be implementable today, I think? If numBroadcastPeers > 0 post 0.10.3 then you know the tx made it out to the internet. Unfortunately if nodes start to diverge a lot in terms of what they will accept, then “transmitted” is no longer a clean binary yes/no thing. Guess we’ll have to jump that hurdle when we come to it. > Guess you're right. But as you said, we're not there yet. The payment protocol at least would need some notion of fee, or possibly (better?) the ability for a recipient to specify some inputs as well as some outputs. Originally I think we were hoping for child-pays-for-parent. I guess that needs someone to sit down and focus on it for a while, assuming we still think that’s a good idea. smime.p7s Description: S/MIME cryptographic signature -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
On 12/01/2013 06:19 PM, Mike Hearn wrote:
>> Both can be combined into adapting the current generic messages ("This
>> payment should become spendable shortly" for incoming and "This payment
>> has not been transmitted yet" for outgoing transactions).
>
> What would the new messages say?
Well, for starters I'd suggest something like
"This payment did not become spendable since xxx minutes. Check with the
sender if s/he paid the Bitcoin network fee. Check if your internet
connection is working properly." (incoming)
"This payment still has not been transmitted. Check if your internet
connection is working properly." (outgoing)
> We need to get away from the notion of senders attaching fees anyway.
This is the wrong way around because it’s the recipient who cares about
double spending risk, not the sender. That’s why merchants keep running
into issues with people attaching zero fees. Of course they attach zero
fees. They know they aren’t going to double spend. It’s the merchant who
cares about getting the security against that.
Guess you're right. But as you said, we're not there yet.
> The UI for sending money should end up dead simple - no mention of
fees anywhere, IMO.
Agreed, if the sender does not need to pay a fee any more. On the
receiving side it of course needs to be mentioned. (Or the other way
round, as of today.)
> Unfortunately we lack the protocol pieces to get the right UI here :(
Someone needs to sit down and figure out what the UI *should* look like,
in the ideal world, and then work backwards to figure out what needs to
be done to get us there.
(The ideal world doesn't need a UI for money.)
>> For outgoing transactions, if it is very clear that they're never going
>> to be confirmed, I'd like to see a "Revoke" button.
>
> Disagree. There should never be any cases in which a transaction
doesn’t confirm. Period. I know there have been bugs with bitcoinj that
could cause this in the past, but they were bugs and they got fixed/will
get fixed.
>
> Settlement failure is just unacceptable and building a UI around the
possibility will just encourage people to think of it as normal, when it
should not be so.
I fully understand your point of view. However, its not the reality
currently. (Hopefully it is, after the fixes to bitcoinj.)
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
> Both can be combined into adapting the current generic messages ("This
> payment should become spendable shortly" for incoming and "This payment
> has not been transmitted yet" for outgoing transactions).
What would the new messages say?
We need to get away from the notion of senders attaching fees anyway. This is
the wrong way around because it’s the recipient who cares about double spending
risk, not the sender. That’s why merchants keep running into issues with people
attaching zero fees. Of course they attach zero fees. They know they aren’t
going to double spend. It’s the merchant who cares about getting the security
against that.
The UI for sending money should end up dead simple - no mention of fees
anywhere, IMO.
The UI for receiving money could be a bit more complicated but even then - I
think if ordinary people using smartphone wallets are having to think about how
quickly they want their transaction to confirm and adjust fees, etc on the
receiving side then we’re getting dangerously close to the usability failure
zone.
Unfortunately we lack the protocol pieces to get the right UI here :( Someone
needs to sit down and figure out what the UI *should* look like, in the ideal
world, and then work backwards to figure out what needs to be done to get us
there.
> For outgoing transactions, if it is very clear that they're never going
> to be confirmed, I'd like to see a "Revoke" button.
Disagree. There should never be any cases in which a transaction doesn’t
confirm. Period. I know there have been bugs with bitcoinj that could cause
this in the past, but they were bugs and they got fixed/will get fixed.
Settlement failure is just unacceptable and building a UI around the
possibility will just encourage people to think of it as normal, when it should
not be so.
smime.p7s
Description: S/MIME cryptographic signature
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
(my post hasn't shown up for an hour, so I'm sending it again)
On 12/01/2013 02:41 PM, Mike Hearn wrote:
>> As long as the tx is not confirmed (by a broadcast), apps can offer to
>> bump up the fee a little bit.
>
> Unfortunately there are risks to that approach.
I assume you're right, since I do not have so much experience with game
theory.
About the UI:
Generally, for pending tx I'd like to measure time they're not being
broadcast-confirmed and count blocks that they missed being included.
Both can be combined into adapting the current generic messages ("This
payment should become spendable shortly" for incoming and "This payment
has not been transmitted yet" for outgoing transactions). Hint:
Statistics could be offered by bitcoinj.
For outgoing transactions, if it is very clear that they're never going
to be confirmed, I'd like to see a "Revoke" button. This would have
saved us some support hassles with the transmit bugs. It could also
offer a "Top up fee" button, which would replace the tx by a new one.
I'm aware about a possible double spend but who cares? It doesn't matter
which of the two transactions gets into the chain, as long as not both
will be included.
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
> As long as the tx is not confirmed (by a broadcast), apps can offer to > bump up the fee a little bit. Unfortunately there are risks to that approach. The most obvious one is that nodes could keep sending reject messages to get wallets to attach ridiculously high fees. If half a wallets peers do this and the other half don’t, then effectively the wallet will double spend against itself. The bad nodes can keep the fat transaction and send it directly to a corrupt miner, no broadcast. If some other miner includes the original normal transaction, no problem, just take it out of the current block. If the corrupt miner finds the current block, they get to claim huge fee premiums. Quite apart from the problem of malicious nodes/miners, how would you represent this in the wallet GUI? Current wallets are designed on the assumption that 1 payment == 1 transaction == 1 paid fee. If a single payment could have several different fees, and there’s no way to know which you will actually pay until later, then complexity would explode. Even the notion of balance would become even more complicated than it already is. So I really don’t like the idea of creating different transactions depending on error messages from remote nodes. The only time when it could make sense is if *all* nodes reject a transaction. Then (assuming no MITM) you can assume the first transaction can be thrown away and a new attempt made. But if you think about what the UI flows for that would look like - it’s just a mess. There are other risks to fee estimation. Let’s say wallet authors create transactions with exactly the estimated fee needed to get into the next block. But due to mempool skew, estimates vary, and so those transactions don’t propagate cleanly everywhere. Now we have two problems: 1) Unpredictable failure to enter the mempools can lead to double spending and slow confirmations 2) Wallet authors may be tempted to ensure that doesn’t happen by taking the estimate, adding 10% and using that. But then if a bunch of popular wallets all do the same thing, the estimation algorithm might get confused and decide that as everyone seems to be attaching a fee of X+10%, the correct estimate for what fee to attach is X+10%. Then wallets would immediate raise their attached fees again and you’d enter into a infinite upward spiral. The more I think about this, the more complicated it gets. It’s tempting to try and just push all the complexity onto the merchant side, but one of the best things about Bitcoin is there isn’t any strong notion of “merchant” - that’s inherent to being peer to peer. So just hand-waving and saying sellers will deal with complicated fee processes is just a punt. smime.p7s Description: S/MIME cryptographic signature -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
On 12/01/2013 12:51 PM, Mike Hearn wrote: > I propose the following plan: Thanks taking the initiative and writing this up! > If a wallet gets a reject message for a tx that has a fee that are by > its own estimates acceptable, what should it do? What if only some nodes > report that and others don't? As long as the tx is not confirmed (by a broadcast), apps can offer to bump up the fee a little bit. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Floating fees and SPV clients
Lately I was pondering how to make floating fees and SPV wallets work well together. I propose the following plan: 1) 0.9 ships with something dead simple, like a command to query what a node estimates and then clients just take the average, or cross-check a centralised estimate against the P2P network. It's fast to implement and simple, but not very secure or decentralised. However it will allow the feature to launch on some kind of reasonable timeframe. 2) We bump the protocol version and the tx message now gets an optional protobuf buffer stuck on the end. The first thing put in this protobuf is a list of the values of the inputs. Using this data, the fee paid by a transaction can be calculated. In step 2 the data is unauthenticated. 3) Some SPV wallets already set themselves up so that they sync with the network in the background, e.g. the Android wallet syncs at least every 24 hours. This should become more common, using scheduler capabilities built into most operating systems. When the wallet syncs with the network, it sets a deliberately very noisy Bloom filter on its peers and waits around for 30-60 seconds or so. The wallet observes some of the broadcasts taking place and records the hashes and associated fees that were paid to disk. Next time it syncs, it includes the observed hashes into the Bloom filter used to download the chain, and thus learns how quickly they confirmed. It can calculate its own fee estimate from that. 4) Finally, when we next hard fork, we make v2 transactions include the output value in the signature, same as the output script (this proposal has been on the forums for a while now). That allows the fee data added in step 2 to be cross-checked against the signatures on the inputs, thus authenticating it. I think this is a small and easy set of steps that would make it quite hard to attack - malicious nodes could make it appear that some transactions never confirmed thus seeming to force the price up, but it's easy to simply exclude transactions which never confirm at all from the calculations. Plus of course you can cross-check nodes against each other to try and catch nodes that are failing to match transactions properly. One obvious concern is what to do if nodes don't converge on very similar estimates. Wallets will always want to pay the lowest fee possible, so that means they'll always be riding the very edge of what's acceptable, opening up tx propagation to random flaky failures if fee estimates change whilst a transaction is in progress, or if some nodes don't calculate the same estimates as others. If a wallet gets a reject message for a tx that has a fee that are by its own estimates acceptable, what should it do? What if only some nodes report that and others don't? -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Coin Control, Send crash on MacOS X
https://github.com/litecoin-project/bitcoinomg/commits/0.8.5-OMG6 http://download1.rpmfusion.org/~warren/bitcoin-0.8.5-OMG6/ I've been backporting patches from master and Litecoin to make a Bitcoin 0.8 client with more features. It works quite well on Linux and Win32. http://pastebin.com/g8QqheGc Today we discovered a rare crash that can happen on MacOS X. toffoo and coblee reproduced it on MacOS X 10.9 and I reproduced it on 10.6.8. It seems to be some kind of race condition involving SendCoinsEntry::clear(). 1. 11 QtGui 0x00e28141 QWidget::setFocus(Qt::FocusReason) + 289 2. 12 org.bitcoinfoundation.Bitcoin-Qt0x002ca665 SendCoinsEntry::clear() + 101 This build was made with Xcode 3.2.6 on MacOS X with MacPorts qt4-mac qt-4.8.4, roughly meant to approximate Gavin's build environment for the 0.8.x releases. With this unfamiliar build environment I have been unsuccessful at building master so I am unable to confirm if this crash exists there. I am trying qt-4.8.5 next ... but even if I manage to build it, it is exceedingly difficult to reproduce... Warren -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
