Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[Jonas Schnelli via bitcoin-dev]

Hi

> Unconfirmed transactions are incredibly important for real world use.
> Merchants for instance are willing to accept credit card payments of
> thousands of dollars and ship the goods despite the fact that the
> transaction can be reversed up to 60 days later. There is a very large
> cost to losing the ability to have instant transactions in many or
> even most situations. This cost is typically well above the fraud risk. 
>
> It's important to recognize that bitcoin serves a wide variety of use
> cases with different profiles for time sensitivity and fraud risk.
>
I agree that unconfirmed transactions are incredibly important, but not
over SPV against random peers.

If you offer users/merchants a feature (SPV 0-conf against random
peers), that is fundamentally insecure, it will – sooner or later – lead
to some large scale fiasco, hurting Bitcoins reputation and trust from
merchants.

Merchants using and trusting 0-conf SPV transactions (retrieved from
random peers) is something we should **really eliminate** through
education and by offering different solution.

There are plenty, more sane options. If you can't run your own full-node
as a merchant (trivial), maybe co-use a wallet-service with centralized
verification (maybe use two of them), I guess Copay would be one of
those wallets (as an example). Use them in watch-only mode.

For end-users SPV software, I think it would be recommended to...
... disable unconfirmed transactions during SPV against random peers
... enable unconfirmed transactions when using SPV against a trusted
peer with preshared keys after BIP150
... if unconfirmed transactions are disabled, show how it can be enabled
(how to run a full-node [in a box, etc.])
... educate, inform users that a transaction with no confirmation can be
"stopped" or "redirected" any time, also inform about the risks during
low-conf phase (1-5).

I though see the point that it's nice to make use of the "incoming
funds..." feature in SPV wallets. But – for the sake of stability and
(risk-)scaling – we may want to recommend to scarify this feature and –
in the same turn – to use privacy-preserving BFD's.






signature.asc
Description: OpenPGP digital signature
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[Eric Voskuil via bitcoin-dev]

Credit card reversals involve an escrow agent with control over the entire 
network and with a strong interest in preserving the network. A better analogy 
would be blind acceptance of any slip of paper under the assumption that it is 
sufficient currency. It may or may not be so, but you are on your own in either 
case.

e

> On Jan 3, 2017, at 4:36 PM, Aaron Voisine via bitcoin-dev 
>  wrote:
> 
> Knowing that a transaction is property formatted and that it has been 
> broadcast to the gossip network is useful in many situations. You're only 
> thinking about whether you can know a transaction is valid and/or settled. 
> This is not the only possible useful information in actual real world use. 
> Any situation where credit card transactions are accepted today for instance, 
> it is useful to know that a transaction has been initiated, even though it 
> can be reversed at any time up to 60 days later.
> 
> Aaron Voisine
> co-founder and CEO
> breadwallet
> 
>> On Tue, Jan 3, 2017 at 4:10 PM,  wrote:
>> Unfortunately a non validating SPV wallet has absolutely no idea if
>> the information about an unconfirmed transaction they are seeing is
>> anything but properly formatted. They are connecting to an easily
>> manipulated, sybil attacked, and untrusted network and then asking
>> them for financial information. Seeing an unconfirmed transaction in a
>> wallet that's not also fully validating is at best meaningless.
>> 
>> 
>>> On 2017-01-03 15:46, Aaron Voisine wrote:
>>> If the sender doesn't control the receiver's network connection, then
>>> the information the receiver gains by watching the mempool is if the
>>> transaction has propagated across the bitcoin network. This is useful
>>> to know in all kinds of situations.
>>> 
>>> Aaron Voisine
>>> co-founder and CEO
>>> breadwallet [2]
>>> 
>>> On Tue, Jan 3, 2017 at 3:06 PM, adiabat  wrote:
>>> 
 Mempool transactions have their place, but "unconfirmed" and "SPV"
 don't belong together.  Only a full node can tell if a transaction
 may get confirmed, or is nonsense.  Unfortunately all the light /
 SPV wallets I know of show mempool transactions, which makes it hard
 to go back... (e.g. "why doesn't your software show 0-conf! your
 wallet is broken!", somewhat akin to people complaining about RBF)
 
 So, this is easy, just don't worry about mempool filtering.  Why are
 light clients looking at the mempool anyway?  Maybe if there were
 some way to provide SPV proofs of all inputs, but that's a bit of a
 mess for full nodes to do.
 
 Without mempool filtering, I think the committed bloom filters would
 be a great improvement over the current bloom filter setup,
 especially for lightning network use cases (with lightning, not
 finding out about a transaction can make you lose money).  I want to
 work on it and may be able to at some point as it's somewhat related
 to lightning.
 
 Also, if you're running a light client, and storing the filters the
 way you store block headers, there's really no reason to go all the
 way back to height 0.  You can start grabbing headers at some point
 a while ago, before your set of keys was generated.  I think it'd be
 very worth it even with GB-scale disk usage.
 
 -Tadge
 
 On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev
  wrote:
 
 Unconfirmed transactions are incredibly important for real world
 use. Merchants for instance are willing to accept credit card
 payments of thousands of dollars and ship the goods despite the fact
 that the transaction can be reversed up to 60 days later. There is a
 very large cost to losing the ability to have instant transactions
 in many or even most situations. This cost is typically well above
 the fraud risk.
 
 It's important to recognize that bitcoin serves a wide variety of
 use cases with different profiles for time sensitivity and fraud
 risk.
 
 Aaron
 
 On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev
  wrote:
 The concept combined with the weak blocks system where miners commit
 
 to potential transaction inclusion with fractional difficulty blocks
 
 is possible. I'm not personally convinced that unconfirmed
 transaction
 
 display in a wallet is worth the privacy trade-off. The user has
 very
 
 little to gain from this knowledge until the txn is in a block.
 
 On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:
 
> Hi
 
>> We introduce several concepts that rework the lightweight Bitcoin
 
>> client model in a manner which is secure, efficient and privacy
 
>> compatible.
 
>> 
 
>> The BFD can be used verbatim in 

Re: [bitcoin-dev] Script Abuse Potential?

[Russell O'Connor via bitcoin-dev]

For the record, the OP_CAT limit of 520 bytes was added by Satoshi

on the famous August 15, 2010 "misc" commit, at the same time that OP_CAT
was disabled.
The previous limit was 5000 bytes.

On Tue, Jan 3, 2017 at 7:13 PM, Jeremy via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Sure, was just upper bounding it anyways. Even less of a problem!
>
>
> RE: OP_CAT, not as OP_CAT was specified, which is why it was disabled. As
> far as I know, the elements alpha proposal to reenable a limited op_cat to
> 520 bytes is somewhat controversial...
>
>
>
> --
> @JeremyRubin 
> 
>
> On Mon, Jan 2, 2017 at 10:39 PM, Johnson Lau  wrote:
>
>> No, there could only have not more than 201 opcodes in a script. So you
>> may have 198 OP_2DUP at most, i.e. 198 * 520 * 2 = 206kB
>>
>> For OP_CAT, just check if the returned item is within the 520 bytes limit.
>>
>> On 3 Jan 2017, at 11:27, Jeremy via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> It is an unfortunate script, but can't actually
>> ​do
>>  that much
>> ​ it seems​
>> . The MAX_SCRIPT_ELEMENT_SIZE = 520 Bytes.
>> ​ Thus, it would seem the worst you could do with this would be to 
>> (1-520*2)*520*2
>> bytes  ~=~ 10 MB.
>>
>> ​Much more concerning would be the op_dup/op_cat style bug, which under a
>> similar script ​would certainly cause out of memory errors :)
>>
>>
>>
>> --
>> @JeremyRubin 
>> 
>>
>> On Mon, Jan 2, 2017 at 4:39 PM, Steve Davis via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>> Hi all,
>>>
>>> Suppose someone were to use the following pk_script:
>>>
>>> [op_2dup, op_2dup, op_2dup, op_2dup, op_2dup, ...(to limit)...,
>>> op_2dup, op_hash160, , op_equalverify, op_checksig]
>>>
>>> This still seems to be valid AFAICS, and may be a potential attack
>>> vector?
>>>
>>> Thanks.
>>>
>>>
>>> ___
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>>
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>>
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[Aaron Voisine via bitcoin-dev]

Knowing that a transaction is property formatted and that it has been
broadcast to the gossip network is useful in many situations. You're only
thinking about whether you can know a transaction is valid and/or settled.
This is not the only possible useful information in actual real world use.
Any situation where credit card transactions are accepted today for
instance, it is useful to know that a transaction has been initiated, even
though it can be reversed at any time up to 60 days later.

Aaron Voisine
co-founder and CEO
breadwallet 

On Tue, Jan 3, 2017 at 4:10 PM,  wrote:

> Unfortunately a non validating SPV wallet has absolutely no idea if
> the information about an unconfirmed transaction they are seeing is
> anything but properly formatted. They are connecting to an easily
> manipulated, sybil attacked, and untrusted network and then asking
> them for financial information. Seeing an unconfirmed transaction in a
> wallet that's not also fully validating is at best meaningless.
>
>
> On 2017-01-03 15:46, Aaron Voisine wrote:
>
>> If the sender doesn't control the receiver's network connection, then
>> the information the receiver gains by watching the mempool is if the
>> transaction has propagated across the bitcoin network. This is useful
>> to know in all kinds of situations.
>>
>> Aaron Voisine
>> co-founder and CEO
>> breadwallet [2]
>>
>> On Tue, Jan 3, 2017 at 3:06 PM, adiabat  wrote:
>>
>> Mempool transactions have their place, but "unconfirmed" and "SPV"
>>> don't belong together.  Only a full node can tell if a transaction
>>> may get confirmed, or is nonsense.  Unfortunately all the light /
>>> SPV wallets I know of show mempool transactions, which makes it hard
>>> to go back... (e.g. "why doesn't your software show 0-conf! your
>>> wallet is broken!", somewhat akin to people complaining about RBF)
>>>
>>> So, this is easy, just don't worry about mempool filtering.  Why are
>>> light clients looking at the mempool anyway?  Maybe if there were
>>> some way to provide SPV proofs of all inputs, but that's a bit of a
>>> mess for full nodes to do.
>>>
>>> Without mempool filtering, I think the committed bloom filters would
>>> be a great improvement over the current bloom filter setup,
>>> especially for lightning network use cases (with lightning, not
>>> finding out about a transaction can make you lose money).  I want to
>>> work on it and may be able to at some point as it's somewhat related
>>> to lightning.
>>>
>>> Also, if you're running a light client, and storing the filters the
>>> way you store block headers, there's really no reason to go all the
>>> way back to height 0.  You can start grabbing headers at some point
>>> a while ago, before your set of keys was generated.  I think it'd be
>>> very worth it even with GB-scale disk usage.
>>>
>>> -Tadge
>>>
>>> On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev
>>>  wrote:
>>>
>>> Unconfirmed transactions are incredibly important for real world
>>> use. Merchants for instance are willing to accept credit card
>>> payments of thousands of dollars and ship the goods despite the fact
>>> that the transaction can be reversed up to 60 days later. There is a
>>> very large cost to losing the ability to have instant transactions
>>> in many or even most situations. This cost is typically well above
>>> the fraud risk.
>>>
>>> It's important to recognize that bitcoin serves a wide variety of
>>> use cases with different profiles for time sensitivity and fraud
>>> risk.
>>>
>>> Aaron
>>>
>>> On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev
>>>  wrote:
>>> The concept combined with the weak blocks system where miners commit
>>>
>>> to potential transaction inclusion with fractional difficulty blocks
>>>
>>> is possible. I'm not personally convinced that unconfirmed
>>> transaction
>>>
>>> display in a wallet is worth the privacy trade-off. The user has
>>> very
>>>
>>> little to gain from this knowledge until the txn is in a block.
>>>
>>> On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:
>>>
>>> Hi

>>>
>>> We introduce several concepts that rework the lightweight Bitcoin
>

>>> client model in a manner which is secure, efficient and privacy
>

>>> compatible.
>

>>>
>
>>> The BFD can be used verbatim in replacement of BIP37, where the
>
 filter
>>>
>>> can be cached between clients without needing to be recomputed.
>
 It can
>>>
>>> also be used by normal pruned nodes to do re-scans locally of
>
 their
>>>
>>> wallet without needing to have the block data available to scan,
>
 or
>>>
>>> without reading the entire block chain from disk.
>

>>> I started exploring the potential of BFD after this specification.

>>>
>>>

>>> What would be the preferred/recommended way to handle

>>> 0-conf/mempool

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[bfd--- via bitcoin-dev]

Unfortunately a non validating SPV wallet has absolutely no idea if
the information about an unconfirmed transaction they are seeing is
anything but properly formatted. They are connecting to an easily
manipulated, sybil attacked, and untrusted network and then asking
them for financial information. Seeing an unconfirmed transaction in a
wallet that's not also fully validating is at best meaningless.


On 2017-01-03 15:46, Aaron Voisine wrote:

If the sender doesn't control the receiver's network connection, then
the information the receiver gains by watching the mempool is if the
transaction has propagated across the bitcoin network. This is useful
to know in all kinds of situations.

Aaron Voisine
co-founder and CEO
breadwallet [2]
On Tue, Jan 3, 2017 at 3:06 PM, adiabat  wrote:


Mempool transactions have their place, but "unconfirmed" and "SPV"
don't belong together.  Only a full node can tell if a transaction
may get confirmed, or is nonsense.  Unfortunately all the light /
SPV wallets I know of show mempool transactions, which makes it hard
to go back... (e.g. "why doesn't your software show 0-conf! your
wallet is broken!", somewhat akin to people complaining about RBF)

So, this is easy, just don't worry about mempool filtering.  Why are
light clients looking at the mempool anyway?  Maybe if there were
some way to provide SPV proofs of all inputs, but that's a bit of a
mess for full nodes to do.

Without mempool filtering, I think the committed bloom filters would
be a great improvement over the current bloom filter setup,
especially for lightning network use cases (with lightning, not
finding out about a transaction can make you lose money).  I want to
work on it and may be able to at some point as it's somewhat related
to lightning.

Also, if you're running a light client, and storing the filters the
way you store block headers, there's really no reason to go all the
way back to height 0.  You can start grabbing headers at some point
a while ago, before your set of keys was generated.  I think it'd be
very worth it even with GB-scale disk usage.

-Tadge

On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev
 wrote:

Unconfirmed transactions are incredibly important for real world
use. Merchants for instance are willing to accept credit card
payments of thousands of dollars and ship the goods despite the fact
that the transaction can be reversed up to 60 days later. There is a
very large cost to losing the ability to have instant transactions
in many or even most situations. This cost is typically well above
the fraud risk.

It's important to recognize that bitcoin serves a wide variety of
use cases with different profiles for time sensitivity and fraud
risk.

Aaron

On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev
 wrote:
The concept combined with the weak blocks system where miners commit

to potential transaction inclusion with fractional difficulty blocks

is possible. I'm not personally convinced that unconfirmed
transaction

display in a wallet is worth the privacy trade-off. The user has
very

little to gain from this knowledge until the txn is in a block.

On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:


Hi



We introduce several concepts that rework the lightweight Bitcoin



client model in a manner which is secure, efficient and privacy



compatible.







The BFD can be used verbatim in replacement of BIP37, where the

filter


can be cached between clients without needing to be recomputed.

It can


also be used by normal pruned nodes to do re-scans locally of

their


wallet without needing to have the block data available to scan,

or


without reading the entire block chain from disk.



I started exploring the potential of BFD after this specification.







What would be the preferred/recommended way to handle

0-conf/mempool


filtering – if & once BDF would have been deployed (any type,



semi-trusted oracles or protocol-level/softfork)?







From the user-experience perspective, this is probably pretty

important


(otherwise the experience will be that incoming funds can take

serval


minutes to hours until they appear).



Using BIP37 bloom filters just for mempool filtering would

obviously


result in the same unwanted privacy-setup.



















___



bitcoin-dev mailing list



bitcoin-dev@lists.linuxfoundation.org



https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1]


___

bitcoin-dev mailing list

bitcoin-dev@lists.linuxfoundation.org

https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1]

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1]




Links:
--
[1] 

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[Aaron Voisine via bitcoin-dev]

If the sender doesn't control the receiver's network connection, then the
information the receiver gains by watching the mempool is if the
transaction has propagated across the bitcoin network. This is useful to
know in all kinds of situations.


Aaron Voisine
co-founder and CEO
breadwallet 

On Tue, Jan 3, 2017 at 3:06 PM, adiabat  wrote:

> Mempool transactions have their place, but "unconfirmed" and "SPV" don't
> belong together.  Only a full node can tell if a transaction may get
> confirmed, or is nonsense.  Unfortunately all the light / SPV wallets I
> know of show mempool transactions, which makes it hard to go back... (e.g.
> "why doesn't your software show 0-conf! your wallet is broken!", somewhat
> akin to people complaining about RBF)
>
> So, this is easy, just don't worry about mempool filtering.  Why are light
> clients looking at the mempool anyway?  Maybe if there were some way to
> provide SPV proofs of all inputs, but that's a bit of a mess for full nodes
> to do.
>
> Without mempool filtering, I think the committed bloom filters would be a
> great improvement over the current bloom filter setup, especially for
> lightning network use cases (with lightning, not finding out about a
> transaction can make you lose money).  I want to work on it and may be able
> to at some point as it's somewhat related to lightning.
>
> Also, if you're running a light client, and storing the filters the way
> you store block headers, there's really no reason to go all the way back to
> height 0.  You can start grabbing headers at some point a while ago, before
> your set of keys was generated.  I think it'd be very worth it even with
> GB-scale disk usage.
>
> -Tadge
>
>
> On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Unconfirmed transactions are incredibly important for real world use.
>> Merchants for instance are willing to accept credit card payments of
>> thousands of dollars and ship the goods despite the fact that the
>> transaction can be reversed up to 60 days later. There is a very large cost
>> to losing the ability to have instant transactions in many or even most
>> situations. This cost is typically well above the fraud risk.
>>
>> It's important to recognize that bitcoin serves a wide variety of use
>> cases with different profiles for time sensitivity and fraud risk.
>>
>> Aaron
>>
>> On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>> The concept combined with the weak blocks system where miners commit
>>>
>>> to potential transaction inclusion with fractional difficulty blocks
>>>
>>> is possible. I'm not personally convinced that unconfirmed transaction
>>>
>>> display in a wallet is worth the privacy trade-off. The user has very
>>>
>>> little to gain from this knowledge until the txn is in a block.
>>>
>>>
>>>
>>>
>>>
>>> On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:
>>>
>>> > Hi
>>>
>>> >> We introduce several concepts that rework the lightweight Bitcoin
>>>
>>> >> client model in a manner which is secure, efficient and privacy
>>>
>>> >> compatible.
>>>
>>> >>
>>>
>>> >> The BFD can be used verbatim in replacement of BIP37, where the filter
>>>
>>> >> can be cached between clients without needing to be recomputed. It can
>>>
>>> >> also be used by normal pruned nodes to do re-scans locally of their
>>>
>>> >> wallet without needing to have the block data available to scan, or
>>>
>>> >> without reading the entire block chain from disk.
>>>
>>> > I started exploring the potential of BFD after this specification.
>>>
>>> >
>>>
>>> > What would be the preferred/recommended way to handle 0-conf/mempool
>>>
>>> > filtering – if & once BDF would have been deployed (any type,
>>>
>>> > semi-trusted oracles or protocol-level/softfork)?
>>>
>>> >
>>>
>>> > From the user-experience perspective, this is probably pretty important
>>>
>>> > (otherwise the experience will be that incoming funds can take serval
>>>
>>> > minutes to hours until they appear).
>>>
>>> > Using BIP37 bloom filters just for mempool filtering would obviously
>>>
>>> > result in the same unwanted privacy-setup.
>>>
>>> >
>>>
>>> > 
>>>
>>> >
>>>
>>> >
>>>
>>> > ___
>>>
>>> > bitcoin-dev mailing list
>>>
>>> > bitcoin-dev@lists.linuxfoundation.org
>>>
>>> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>> ___
>>>
>>> bitcoin-dev mailing list
>>>
>>> bitcoin-dev@lists.linuxfoundation.org
>>>
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>>
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>
___
bitcoin-dev mailing list

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[bfd--- via bitcoin-dev]

The concept was not particularly targeted towards businesses, but
allowing for significantly improved wallet performance and reducing
privacy for lite clients. You would expect that a business has the
capacity to run a fully validating, fully storing node of their own.
If they’re not something is fundamentally broken with Bitcoin, or
their rationale of continuing to use it.


On 2017-01-03 14:18, Aaron Voisine wrote:

Unconfirmed transactions are incredibly important for real world use.
Merchants for instance are willing to accept credit card payments of
thousands of dollars and ship the goods despite the fact that the
transaction can be reversed up to 60 days later. There is a very large
cost to losing the ability to have instant transactions in many or
even most situations. This cost is typically well above the fraud
risk.

It's important to recognize that bitcoin serves a wide variety of use
cases with different profiles for time sensitivity and fraud risk.

Aaron

On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev
 wrote:


The concept combined with the weak blocks system where miners commit

to potential transaction inclusion with fractional difficulty blocks

is possible. I'm not personally convinced that unconfirmed
transaction

display in a wallet is worth the privacy trade-off. The user has
very

little to gain from this knowledge until the txn is in a block.

On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:


Hi



We introduce several concepts that rework the lightweight Bitcoin



client model in a manner which is secure, efficient and privacy



compatible.







The BFD can be used verbatim in replacement of BIP37, where the

filter


can be cached between clients without needing to be recomputed.

It can


also be used by normal pruned nodes to do re-scans locally of

their


wallet without needing to have the block data available to scan,

or


without reading the entire block chain from disk.



I started exploring the potential of BFD after this specification.







What would be the preferred/recommended way to handle

0-conf/mempool


filtering – if & once BDF would have been deployed (any type,



semi-trusted oracles or protocol-level/softfork)?







From the user-experience perspective, this is probably pretty

important


(otherwise the experience will be that incoming funds can take

serval


minutes to hours until they appear).



Using BIP37 bloom filters just for mempool filtering would

obviously


result in the same unwanted privacy-setup.



















___



bitcoin-dev mailing list



bitcoin-dev@lists.linuxfoundation.org



https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


___

bitcoin-dev mailing list

bitcoin-dev@lists.linuxfoundation.org

https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[Aaron Voisine via bitcoin-dev]

Unconfirmed transactions are incredibly important for real world use.
Merchants for instance are willing to accept credit card payments of
thousands of dollars and ship the goods despite the fact that the
transaction can be reversed up to 60 days later. There is a very large cost
to losing the ability to have instant transactions in many or even most
situations. This cost is typically well above the fraud risk.

It's important to recognize that bitcoin serves a wide variety of use cases
with different profiles for time sensitivity and fraud risk.

Aaron

On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> The concept combined with the weak blocks system where miners commit
>
> to potential transaction inclusion with fractional difficulty blocks
>
> is possible. I'm not personally convinced that unconfirmed transaction
>
> display in a wallet is worth the privacy trade-off. The user has very
>
> little to gain from this knowledge until the txn is in a block.
>
>
>
>
>
> On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:
>
> > Hi
>
> >> We introduce several concepts that rework the lightweight Bitcoin
>
> >> client model in a manner which is secure, efficient and privacy
>
> >> compatible.
>
> >>
>
> >> The BFD can be used verbatim in replacement of BIP37, where the filter
>
> >> can be cached between clients without needing to be recomputed. It can
>
> >> also be used by normal pruned nodes to do re-scans locally of their
>
> >> wallet without needing to have the block data available to scan, or
>
> >> without reading the entire block chain from disk.
>
> > I started exploring the potential of BFD after this specification.
>
> >
>
> > What would be the preferred/recommended way to handle 0-conf/mempool
>
> > filtering – if & once BDF would have been deployed (any type,
>
> > semi-trusted oracles or protocol-level/softfork)?
>
> >
>
> > From the user-experience perspective, this is probably pretty important
>
> > (otherwise the experience will be that incoming funds can take serval
>
> > minutes to hours until they appear).
>
> > Using BIP37 bloom filters just for mempool filtering would obviously
>
> > result in the same unwanted privacy-setup.
>
> >
>
> > 
>
> >
>
> >
>
> > ___
>
> > bitcoin-dev mailing list
>
> > bitcoin-dev@lists.linuxfoundation.org
>
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
> ___
>
> bitcoin-dev mailing list
>
> bitcoin-dev@lists.linuxfoundation.org
>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[bfd--- via bitcoin-dev]

The concept combined with the weak blocks system where miners commit
to potential transaction inclusion with fractional difficulty blocks
is possible. I'm not personally convinced that unconfirmed transaction
display in a wallet is worth the privacy trade-off. The user has very
little to gain from this knowledge until the txn is in a block.


On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:

Hi

We introduce several concepts that rework the lightweight Bitcoin
client model in a manner which is secure, efficient and privacy
compatible.

The BFD can be used verbatim in replacement of BIP37, where the filter
can be cached between clients without needing to be recomputed. It can
also be used by normal pruned nodes to do re-scans locally of their
wallet without needing to have the block data available to scan, or
without reading the entire block chain from disk.

I started exploring the potential of BFD after this specification.

What would be the preferred/recommended way to handle 0-conf/mempool
filtering – if & once BDF would have been deployed (any type,
semi-trusted oracles or protocol-level/softfork)?

From the user-experience perspective, this is probably pretty important
(otherwise the experience will be that incoming funds can take serval
minutes to hours until they appear).
Using BIP37 bloom filters just for mempool filtering would obviously
result in the same unwanted privacy-setup.




___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security

[bfd--- via bitcoin-dev]

I believe the filter can be more compact than this, but even if not an
order of magnitude saving of disk space is still significant.


On 2016-05-11 13:29, Bob McElrath wrote:
Eelet me revise that last paragraph.  That's 12 *GB* of filters 
at
today's block height (at fixed false-positive rate 1e-6.  Compared to 
block
headers only which are about 33 MB today.  So this proposal is not 
really

compatible with such a wallet being "light"...

Damn units...

Bob McElrath via bitcoin-dev [bitcoin-dev@lists.linuxfoundation.org] 
wrote:

I like this idea, but let's run some numbers...

bfd--- via bitcoin-dev [bitcoin-dev@lists.linuxfoundation.org] wrote:
> A Bloom Filter Digest is deterministically created of every block

Bloom filters completely obfuscate the required size of the filter for 
a desired
false-positive rate.  But, an optimal filter is linear in the number 
of elements
it contains for fixed false-positive rate, and logarithmic in the 
false-positive
rate.  (This comment applies to a RLL encoded Bloom filter Greg 
mentioned, but
that's not the only way)  That is for N elements and false positive 
rate

\epsilon:

filter size = - N \log_2 \epsilon

Given that the data that would be put into this particular filter is 
*already*
hashed, it makes more sense and is faster to use a Cuckoo[1] filter, 
choosing a
fixed false-positive rate, given expected wallet sizes.  For Bloom 
filters,

multiply the above formula by 1.44.

To prevent light clients from downloading more blocks than necessary, 
the
false-positive rate should be roughly less than 1/(block height).  If 
we take
the false positive rate to be 1e-6 for today's block height ~ 41, 
this is
about 20 bits per element.  So for todays block's, this is a 30kb 
filter, for a
3% increase in block size, if blocks commit to the filter.  Thus the 
required

size of the filter commitment is roughly:

filter size = N \log_2 H

where H is the block height.  If bitcoin had these filters from the 
beginning, a
light client today would have to download about 12MB of data in 
filters.  My
personal SPV wallet is using 31MB currently.  It's not clear this is a 
bandwidth

win, though it's definitely a win for computing load on full nodes.


[1] https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf

--
Cheers, Bob McElrath

"For every complex problem, there is a solution that is simple, neat, 
and wrong."

-- H. L. Mencken



!DSPAM:5733934b206851108912031!





___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


!DSPAM:5733934b206851108912031!


--
Cheers, Bob McElrath

"For every complex problem, there is a solution that is simple, neat,
and wrong."
-- H. L. Mencken

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Bitcoin Core 0.13.2 released

[Wladimir J. van der Laan via bitcoin-dev]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Bitcoin Core version 0.13.2 is now available from:

  

Or by bittorrent:

  
magnet:?xt=urn:btih:746697d03db3ff531158b1133bab5d1e4cef4e5a=bitcoin-core-0.13.2=udp%3A%2F%2Ftracker.openbittorrent.com%3A80%2Fannounce=udp%3A%2F%2Ftracker.publicbt.com%3A80%2Fannounce=udp%3A%2F%2Ftracker.ccc.de%3A80%2Fannounce=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969=https%3A%2F%2Fbitcoin.org%2Fbin%2F

This is a new minor version release, including various bugfixes and
performance improvements, as well as updated translations.

Please report bugs using the issue tracker at github:

  

To receive security and update notifications, please subscribe to:

  

Compatibility
==

Microsoft ended support for Windows XP on [April 8th, 
2014](https://www.microsoft.com/en-us/WindowsForBusiness/end-of-xp-support),
an OS initially released in 2001. This means that not even critical security
updates will be released anymore. Without security updates, using a bitcoin
wallet on a XP machine is irresponsible at least.

In addition to that, with 0.12.x there have been varied reports of Bitcoin Core
randomly crashing on Windows XP. It is [not 
clear](https://github.com/bitcoin/bitcoin/issues/7681#issuecomment-217439891)
what the source of these crashes is, but it is likely that upstream
libraries such as Qt are no longer being tested on XP.

We do not have time nor resources to provide support for an OS that is
end-of-life. From 0.13.0 on, Windows XP is no longer supported. Users are
suggested to upgrade to a newer version of Windows, or install an alternative OS
that is supported.

No attempt is made to prevent installing or running the software on Windows XP,
you can still do so at your own risk, but do not expect it to work: do not
report issues about Windows XP to the issue tracker.

- From 0.13.1 onwards OS X 10.7 is no longer supported. 0.13.0 was intended to 
work on 10.7+, 
but severe issues with the libc++ version on 10.7.x keep it from running 
reliably. 
0.13.1 now requires 10.8+, and will communicate that to 10.7 users, rather than 
crashing unexpectedly.

Notable changes
===

Change to wallet handling of mempool rejection
- ---

When a newly created transaction failed to enter the mempool due to
the limits on chains of unconfirmed transactions the sending RPC
calls would return an error.  The transaction would still be queued
in the wallet and, once some of the parent transactions were
confirmed, broadcast after the software was restarted.

This behavior has been changed to return success and to reattempt
mempool insertion at the same time transaction rebroadcast is
attempted, avoiding a need for a restart.

Transactions in the wallet which cannot be accepted into the mempool
can be abandoned with the previously existing abandontransaction RPC
(or in the GUI via a context menu on the transaction).


0.13.2 Change log
=

Detailed release notes follow. This overview includes changes that affect
behavior, not code moves, refactors and string updates. For convenience in 
locating
the code changes and accompanying discussion, both the pull request and
git merge commit are mentioned.

### Consensus
- - #9293 `e591c10` [0.13 Backport #9053] IBD using chainwork instead of height 
and not using header timestamp (gmaxwell)
- - #9053 `5b93eee` IBD using chainwork instead of height and not using header 
timestamps (gmaxwell)

### RPC and other APIs
- - #8845 `1d048b9` Don't return the address of a P2SH of a P2SH (jnewbery)
- - #9041 `87fbced` keypoololdest denote Unix epoch, not GMT (s-matthew-english)
- - #9122 `f82c81b` fix getnettotals RPC description about timemillis 
(visvirial)
- - #9042 `5bcb05d` [rpc] ParseHash: Fail when length is not 64 (MarcoFalke)
- - #9194 `f26dab7` Add option to return non-segwit serialization via rpc 
(instagibbs)
- - #9347 `b711390` [0.13.2] wallet/rpc backports (MarcoFalke)
- - #9292 `c365556` Complain when unknown rpcserialversion is specified (sipa)
- - #9322 `49a612f` [qa] Don't set unknown rpcserialversion (MarcoFalke)

### Block and transaction handling
- - #8357 `ce0d817` [mempool] Fix relaypriority calculation error (maiiz)
- - #9267 `0a4aa87` [0.13 backport #9239] Disable fee estimates for a confirm 
target of 1 block (morcos)
- - #9196 `0c09d9f` Send tip change notification from invalidateblock 
(ryanofsky)

### P2P protocol and network code
- - #8995 `9ef3875` Add missing cs_main lock to ::GETBLOCKTXN processing 
(TheBlueMatt)
- - #9234 `94531b5` torcontrol: Explicitly request RSA1024 private key (laanwj)
- - #8637 `2cad5db` Compact Block Tweaks (rebase of #8235) (sipa)
- - #9058 `286e548` Fixes for p2p-compactblocks.py test timeouts on travis 
(#8842) (ryanofsky)
- - #8865