Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-20 Thread Adam Back via bitcoin-dev
Also Jonas Nick gave a fairly comprehensive presentation on privacy leaks in bitcoin protocol including SPV and bloom query problem specifics: https://www.youtube.com/watch?v=HScK4pkDNds Adam On 20 June 2017 at 14:08, bfd--- via bitcoin-dev wrote: > On

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-20 Thread bfd--- via bitcoin-dev
On 2017-06-20 12:52, Tom Zander via bitcoin-dev wrote: Second, stating that a bloom filter is a "total loss of privacy" is equally baseless and doesn’t need debunking. "On the Privacy Provisions of Bloom Filters in Lightweight Bitcoin Clients" We show analytically and empirically that the

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-20 Thread Tom Zander via bitcoin-dev
On Tuesday, 20 June 2017 00:41:49 CEST Gregory Maxwell via bitcoin-dev wrote: > Can someone make a case why saving no more than those figures would > justify the near total loss of privacy that filtering gives? First, your figures are wrong and also fall out of the sky with no justification.

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-20 Thread Eric Voskuil via bitcoin-dev
The reason that BIP37 presents a long list of problems is that it is a client-server scenario wedged into a peer-to-peer network. The only possible excuse for this design was implementation shortcut. As this thread and others demonstrate, reproducing this design flaw will not eliminate the

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Andreas Schildbach via bitcoin-dev
Most SPV wallets make it quite clear that unconfirmed transactions are just that. On 06/19/2017 06:36 PM, adiabat via bitcoin-dev wrote: > This has been brought up several times in the past, and I agree with > Jonas' comments about users being unaware of the privacy losses due to > BIP37. One

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Tom Zander via bitcoin-dev
On Monday, 19 June 2017 18:30:18 CEST Jonas Schnelli wrote: > I personally would ALWAYS [snip] I mentioned that it was a usability point for a reason, and your personal behavior makes me want to quote one of the main UX guidelines; “You are not your user”

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread adiabat via bitcoin-dev
This has been brought up several times in the past, and I agree with Jonas' comments about users being unaware of the privacy losses due to BIP37. One thing also mentioned before but not int he current thread is that the entire concept of SPV is not applicable to unconfirmed transactions. SPV

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Jonas Schnelli via bitcoin-dev
>> >> I think many users would be willing ... >> a) … to trade higher privacy (using client side filtering) for not having >> the „incoming transaction“ feature b) – if they want 0-conf – to fetch >> all inved transactions > > You seem to misunderstand the usecase. > If you send me a

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Jonas Schnelli via bitcoin-dev
> > On the other hand, I remember only 1 (one) inquiry about the privacy > problems of BIP37 (or privacy at all). IMO privacy its something developers should make sure users have it. Also, I think, todays SPV wallets should make users more aware of the possible privacy implications. Do users

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Tom Zander via bitcoin-dev
On Monday, 19 June 2017 17:49:59 CEST Jonas Schnelli wrote: > Hi > > > On Monday, 19 June 2017 14:26:46 CEST bfd--- via bitcoin-dev wrote: > >> It's been debated if [filtering of] unconfirmed transactions are > >> necessary, > > > > Why would it not be needed? Any SPV client (when used as a > >

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Jonas Schnelli via bitcoin-dev
> Just to give you a number: based on the statistics of the Bitcoin Wallet > app there are at least 2 million wallets depending on BIP37. Not all > would need instant notification but based on the daily support enquiries > instant notificaton is the most asked property of Bitcoin. Yes. Users

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Andreas Schildbach via bitcoin-dev
On 06/19/2017 05:49 PM, Jonas Schnelli via bitcoin-dev wrote: >>> It's been debated if [filtering of] unconfirmed transactions are >>> necessary, >> >> Why would it not be needed? Any SPV client (when used as a payment-receiver) >> requires this from a simple usability point of view. > > > I

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Jonas Schnelli via bitcoin-dev
Hi > On Monday, 19 June 2017 14:26:46 CEST bfd--- via bitcoin-dev wrote: >> It's been debated if [filtering of] unconfirmed transactions are >> necessary, > > Why would it not be needed? Any SPV client (when used as a payment-receiver) > requires this from a simple usability point of view. I

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Andreas Schildbach via bitcoin-dev
Just to give you a number: based on the statistics of the Bitcoin Wallet app there are at least 2 million wallets depending on BIP37. Not all would need instant notification but based on the daily support enquiries instant notificaton is the most asked property of Bitcoin. On 06/19/2017 02:26

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Tom Zander via bitcoin-dev
On Monday, 19 June 2017 14:26:46 CEST bfd--- via bitcoin-dev wrote: > It's been debated if [filtering of] unconfirmed transactions are > necessary, Why would it not be needed? Any SPV client (when used as a payment-receiver) requires this from a simple usability point of view. -- Tom Zander

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread bfd--- via bitcoin-dev
Several times. It's been debated if unconfirmed transactions are necessary, methods of doing more private filtering have been suggested, along with simply not filtering unconfirmed transactions at all. My collected data suggests that there is very little use of BIP37 at present, based on

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-19 Thread Andreas Schildbach via bitcoin-dev
I'm not sure if this has been brought up elsewhere in this thread. This proposal doesn't seem to be a complete replacement of BIP37: It doesn't provide a filter for unconfirmed transactions like BIP37 does. That means that most light clients will continue to use BIP37 even if they may use this

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-09 Thread Tomas via bitcoin-dev
On Fri, Jun 9, 2017, at 05:50, Olaoluwa Osuntokun wrote: > Tomas wrote: > > I don't completely understand the benefit of making the outpoints and > > pubkey hashes (weakly) verifiable. These only serve as notifications and > > therefore do not seem to introduce an attack vector. > > Not sure

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-08 Thread Olaoluwa Osuntokun via bitcoin-dev
> Correct me if I'm wrong, but from my interpretation we can't use that > method as described as we need to output 64-bit integers rather than > 32-bit integers. Had a chat with gmax off-list and came to the realization that the method _should_ indeed generalize to our case of outputting 64-bit

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-08 Thread Olaoluwa Osuntokun via bitcoin-dev
Hi y'all, Thanks for all the comments so far! I've pushed a series of updates to the text of the BIP repo linked in the OP. The fixes include: typos, components of the specification which were incorrect (N is the total number of items, NOT the number of txns in the block), and a few sections

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-08 Thread Olaoluwa Osuntokun via bitcoin-dev
Tomas wrote: > A rough estimate would indicate this to be about 2-2.5x as big per block > as your proposal, but comes with rather different security > characteristics, and would not require download since genesis. Our proposal _doesnt_ require downloading from genesis, if by "downloading" you

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-08 Thread Olaoluwa Osuntokun via bitcoin-dev
Gregory wrote: > I see the inner loop of construction and lookup are free of > non-constant divmod. This will result in implementations being > needlessly slow Ahh, sipa brought this up other day, but I thought he was referring to the coding loop (which uses a power of 2 divisor/modulus), not the

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-08 Thread Olaoluwa Osuntokun via bitcoin-dev
Karl wrote: > I am also curious if you have considered digests containing multiple > blocks. Retaining a permanent binsearchable record of the entire chain is > obviously too space costly, but keeping the last X blocks as binsearchable > could speed up syncing for clients tremendously, I feel.

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-08 Thread Tomas via bitcoin-dev
On Thu, Jun 1, 2017, at 21:01, Olaoluwa Osuntokun via bitcoin-dev wrote:> Hi y'all, > > Alex Akselrod and I would like to propose a new light client BIP for > consideration: >* https://github.com/Roasbeef/bips/blob/master/gcs_light_client.mediawiki> > Very interesting. I would like to

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-07 Thread Gregory Maxwell via bitcoin-dev
On Thu, Jun 1, 2017 at 7:01 PM, Olaoluwa Osuntokun via bitcoin-dev wrote: > Hi y'all, > > Alex Akselrod and I would like to propose a new light client BIP for > consideration: >* https://github.com/Roasbeef/bips/blob/master/gcs_light_client.mediawiki I

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-04 Thread Karl Johan Alm via bitcoin-dev
On Sat, Jun 3, 2017 at 2:55 AM, Alex Akselrod via bitcoin-dev wrote: > Without a soft fork, this is the only way for light clients to verify that > peers aren't lying to them. Clients can request headers (just hashes of the > filters and the previous

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-01 Thread Olaoluwa Osuntokun via bitcoin-dev
> In order to consider the average+median filter sizes in a world worth larger > blocks, I also ran the index for testnet: > > * total size: 2753238530 > * total avg: 5918.95736054141 > * total median: 60202 > * total max: 74983 > * regular size: 1165148878 > * regular

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-01 Thread Alex Akselrod via bitcoin-dev
I agree with Greg and Laolu; BIP-37 filtering for transactions is no better than for blocks and completely destroys privacy. A constant stream of transactions is OK, but even cheaper for light clients would be Laolu's proposal of streaming more tx data than existing inv messages but less than

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-01 Thread Gregory Maxwell via bitcoin-dev
On Fri, Jun 2, 2017 at 2:15 AM, Chris via bitcoin-dev wrote: > On 06/01/2017 06:10 PM, Olaoluwa Osuntokun via bitcoin-dev wrote: > >> One aspect which isn't in this BIP draft is direct support for unconfirmed >> transactions. I consider such a feature an

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-01 Thread Chris via bitcoin-dev
On 06/01/2017 06:10 PM, Olaoluwa Osuntokun via bitcoin-dev wrote: > One aspect which isn't in this BIP draft is direct support for unconfirmed > transactions. I consider such a feature an important UX feature for mobile > phones, and something which I've personally seen as an important >

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-01 Thread Olaoluwa Osuntokun via bitcoin-dev
Eric wrote: > Thanks for sending this proposal! I look forward to having a great > discussion around this. Thanks Eric! We really appreciated the early feedback you gave on the initial design. One aspect which isn't in this BIP draft is direct support for unconfirmed transactions. I consider

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-01 Thread Matt Corallo via bitcoin-dev
Quick comment before I finish reading it completely, looks like you have no way to match the input prevouts being spent, which is rather nice from a "watch for this output being spent" pov. On June 1, 2017 3:01:14 PM EDT, Olaoluwa Osuntokun via bitcoin-dev

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

2017-06-01 Thread Eric Lombrozo via bitcoin-dev
Thanks for sending this proposal! I look forward to having a great discussion around this. - Eric On Thursday, June 1, 2017, Olaoluwa Osuntokun via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Hi y'all, > > Alex Akselrod and I would like to propose a new light client BIP for >