Re: [bitcoin-dev] [BIP Proposal] BetterHash Mining Protocol Replacements

[Matt Corallo via bitcoin-dev]

Clients "inspecting and modifying the transactions" is explicitly *not*
supported. There should be more than enough features for clients to get
bitcoind to generate the exact block they want already available via
Bitcoin Core. The only reason transactions are exposed over the work
protocol at all, really, is so that clients can generate weak blocks to
be sent to the pool for efficient client -> pool block relay, not sure
that's worth bothering to add a whole new endpoint for, sounds
needlessly complicated (and the spec is already more than complicated
enough, sadly).

Matt

On 06/05/18 21:26, Chris Pacia via bitcoin-dev wrote:
> Really like that you're moving forward with this. A few months ago I was
> working on something similar as it seemed like nobody else was interested.
> 
> In regards to the specific proposal, would it make sense to offer a tx
> subscription endpoint in addition to TRANSACTION_DATA_REQUEST? Such an
> endpoint could respond to the subscription with the current full list of
> transactions and then push the diff every time a new template is pushed.
> A client that wants to inspect and modify the transactions would use
> quite a bit less data than polling the request endpoint.
> 
> 
> On 06/05/2018 02:44 PM, Matt Corallo via bitcoin-dev wrote:
>> Been working on this one for a while, so its already been through a few
>> rounds of feeback (thanks to all those who already have provided
>> feedback)!
>>
>> At a high level, this meets a few goals:
>>
>> 1) Replace getblocktemplate with something that is both more performant
>> (no JSON encoding, no full transactions sent over the wire to update a
>> job, hence we can keep the same CTransactionRef in Bitcoin Core making
>> lots of validation things way faster), more robust for consensus changes
>> (no need to add protocol changes to add commitments ala SegWit in the
>> future), and moves more block-switching logic inside of the work
>> provider (allowing Bitcoin Core to better optimize work switching as it
>> knows more than an outside pool server, specifically we can play more
>> games with how we do mempool eviction, empty block mining, and not
>> mining fresh transactions more easily by moving to a more "push" model
>> from the normal "pull" getblocktemplate implementation).
>>
>> 2) Replace Stratum with something more secure (sign messages when
>> applicable, without adding too much overhead to the pool), simpler to
>> implement (not JSON-wrapped-hex, no 32-byte-swapped-per-4-byte-byteorder
>> insanity), and better-defined (a clearly written spec, encompassing the
>> various things shoved backwards into stratum like suggested difficulty
>> in the password field and device identification by setting user to
>> "user.device") with VENDOR_MESSAGEs provided for extensibility instead
>> of conflicting specifications from various different vendors.
>>
>> 3) Provide the ability for a pool to accept work which the users of the
>> pool selected the transactions for, providing strong decentralization
>> pressure by removing the network-level centralization attacks pools can
>> do (or be compromised and used to perform) while still allowing them
>> full control of payout management and variance reduction.
>>
>> While (1) and (2) stand on their own, making it all one set of protocols
>> to provide (3) provides at least the opportunity for drastically better
>> decentralization in Bitcoin mining in the future.
>>
>> The latest version of the full BIP draft can be found at
>> https://github.com/TheBlueMatt/bips/blob/betterhash/bip-.mediawiki
>> and implementations of the work-generation part at
>> https://github.com/TheBlueMatt/bitcoin/commits/2018-02-miningserver and
>> pool/proxy parts at https://github.com/TheBlueMatt/mining-proxy (though
>> note that both implementations are currently on a slightly out-of-date
>> version of the protocol, I hope to get them brought up to date in the
>> coming day or two and make them much more full-featured. The whole stack
>> has managed to mine numerous testnet blocks on several different types
>> of hardware).
>>
>> Matt
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
> 
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Should Graftroot be optional?

[Pieter Wuille via bitcoin-dev]

On Wed, Jun 6, 2018 at 5:48 AM, Tim Ruffing via bitcoin-dev
 wrote:
> On Thu, 2018-05-31 at 17:25 -0700, Pieter Wuille via bitcoin-dev wrote:
>> The best argument for why Graftroot does not need to be optional I
>> think was how Greg put it: "since the signer(s) could have signed an
>> arbitrary transaction instead, being able to delegate is strictly
>> less
>> powerful.".

...

> So
> I think Graftroot delegation is not "strictly less powerful" than just
> using a normal transaction: Graftroot enables to delegate in a way such
> that the delegation itself cannot be fixed in the chain. I think this
> is not possible currently. (Okay, you can just pass around the secret
> keys but has other problems obviously).
>
> Does this have practical implications?
> I don't see any but maybe this helps someone to identify an undesirable
> implication.

Interesting point; I don't see any relevant implications to this
either, but it's indeed good to point out this as a distinction.

> One way to be on the safe side and probably make Greg's argument go
> through is to just define the semantics such that (*) is allowed, i.e.,
> call g-sig a "Graftroot transaction" and give it transaction semantics.
> This provides a new perspective on Graftroot: Then Graftroot does not
> introduce new semantics but (*) is just an optimized version of (**)
> that uses fewer bytes and may be better for privacy.

So you're saying: the Graftroot signature data could be made identical
to the signature hash of an implicit 1-input-1-output transaction
spending the coin and creating a new output with the delegated script
as sPK, and the same amount.

I like that idea, but I don't think it can be *exactly* that. If it's
possible to take a Graftroot signature and instead construct a
transaction with it, you have inherently introduced a malleability.
The created outpoint will be different in both cases (different txid),
meaning that a chain of dependent unconfirmed transactions may be
broken by giving one participant the ability to choose between
Graftroot delegation or actual spending.

Two points here: (1) the implicit transaction would be 0 fee (unless
we somehow assign a portion of the fee to the delegation itself for
purposes of sighash computing), and (2) this sounds very similar to
the issue SIGHASH_NOINPUT is intended to solve. About that...

> Interestingly Andrew's blind-sig example and Johnson's fix (g-sig signs
> the outpoint) are just a special case. If g-sig has transaction
> semantics, it must sign the outpoint (and other stuff).

You're right when you're comparing with existing transaction sighash
semantics, but not when SIGHASH_NOINPUT would exist. If that were the
case, the only real difference is your point above of not being able
to commit the implicit transaction separately. In other words, we're
back to something Johnson pointed out earlier: some of the perceived
problems with Graftroot are also issues with SIGHASH_NOINPUT.

I wonder if we can make this explicit: Graftroot spending becomes a
special sighash flag (which possibly is only allowed at the top level)
- it builds an implicit transaction which moves all the coins to a
newly provided script, computes the sighash of that transaction
(taking all of the Graftroot signature's sighash flags into account -
including potentially SIGHASH_NOINPUT), and requires a signature with
that. The delegated script is then evaluated in the context of that
implicit transaction.

However, in order to avoid the malleability issue I think the actual
signature should still be different - possibly by simply passing
through the Graftroot sighash flag into the sighash being computed.

Cheers,

-- 
Pieter
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size

[Riccardo Casatta via bitcoin-dev]

Sorry if I continue on the subject even if
​custom filter types are considered in BIP 157/158
.
I am doing it
 because
​:
1)​
with a fixed target FP=2^-20  (or 1/784931)
​ and the multi layer filtering maybe it's reasonable to consider less than
~20 bits for the golomb encoding of the per-block filter (one day committed
in the blockchain)
2) based on the answer received, privacy leak if downloading a subset of
filters doesn't look a concern
3)
As far as I know, anyone is considering to use a map instead of a filter
for the upper layers of the filter​.

Simplistic example:
Suppose to have a 2 blocks blockchain, every block contains N items for the
filter:
1) In the current discussed filter we have 2 filters of 20N bits
2) In a two layer solution, we have 1 map of (10+1)2N bits and 2 filters of
10N bits
The additional bit in the map discriminate if the match is in the first or
in the second block.
Supposing to have 1 match in the two blocks, the filter size downloaded in
the first case is always 40N bits, while the expected downloaded size in
the second case is 22N+2^-10*10N+10N ~= 32N with the same FP because
independence.
This obviously isn't a full analysis of the methodology, the expected
downloaded size in the second case could go from the best case 22N bits to
the worst case of 42N bits...

@Gregory
> I don't know what portion of coins created are spent in the same 144 block
window...

About 50%
source code 

>From block 393216 to 458752  (still waiting for results on all the
blockchain)
Total outputs 264185587
size: 2 spent: 11791058 ratio:0.04463172322871649
size: 4 spent: 29846090 ratio:0.11297395266305728
size: 16 spent: 72543182 ratio:0.2745917475051355
size: 64 spent: 113168726 ratio:0.4283682818775424
size: 144 spent: 134294070 ratio:0.508332311103709
size: 256 spent: 148824781 ratio:0.5633342177747191
size: 1024 spent: 179345566 ratio:0.6788620379960395
size: 4096 spent: 205755628 ratio:0.7788298761355213
size: 16384 spent: 224448158 ratio:0.849585174379706

Another point to consider is that if we don't want the full transaction
history of our wallet but only the UTXO, the upper layer map could contain
only the item which are not already spent in the considered window. As we
can see from the previous result if the window is 16384 ~85% of the
elements are already spent suggesting a very high time locality. (apart
144, I choose power of 2 windows so there are an integer number of bits in
the map)

It's possible we need ~20 bits anyway for the per-block filters because
there are always connected wallets which one synced, always download the
last filter, anyway the upper layer map looks very promising for longer
sync.

Il giorno mer 6 giu 2018 alle ore 03:13 Olaoluwa Osuntokun <
laol...@gmail.com> ha scritto:

> It isn't being discussed atm (but was discussed 1 year ago when the BIP
> draft was originally published), as we're in the process of removing items
> or filters that aren't absolutely necessary. We're now at the point where
> there're no longer any items we can remove w/o making the filters less
> generally useful which signals a stopping point so we can begin widespread
> deployment.
>
> In terms of a future extension, BIP 158 already defines custom filter
> types,
> and BIP 157 allows filters to be fetched in batch based on the block height
> and numerical range. The latter feature can later be modified to return a
> single composite filter rather than several individual filters.
>
> -- Laolu
>
>
> On Mon, Jun 4, 2018 at 7:28 AM Riccardo Casatta via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> I was wondering why this multi-layer multi-block filter proposal isn't
>> getting any comment,
>> is it because not asking all filters is leaking information?
>>
>> Thanks
>>
>> Il giorno ven 18 mag 2018 alle ore 08:29 Karl-Johan Alm via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> ha scritto:
>>
>>> On Fri, May 18, 2018 at 12:25 AM, Matt Corallo via bitcoin-dev
>>>  wrote:
>>> > In general, I'm concerned about the size of the filters making existing
>>> > SPV clients less willing to adopt BIP 158 instead of the existing bloom
>>> > filter garbage and would like to see a further exploration of ways to
>>> > split out filters to make them less bandwidth intensive. Some further
>>> > ideas we should probably play with before finalizing moving forward is
>>> > providing filters for certain script templates, eg being able to only
>>> > get outputs that are segwit version X or other similar ideas.
>>>
>>> There is also the idea of multi-block filters. The idea is that light
>>> clients would download a pair of filters for blocks X..X+255 and
>>> X+256..X+511, check if they have any matches and then grab pairs for
>>> any that matched, e.g. X..X+127 & X+128..X+255 if left matched, and
>>> iterate down until it ran out of hits-in-a-row or it got down to
>>> single-block level.
>>>
>>> This has an added benefit where you can accept a s

Re: [bitcoin-dev] Should Graftroot be optional?

[Tim Ruffing via bitcoin-dev]

I haven't read the original Graftroot thread, so maybe all of this has
b
een discussed already or is just wrong... Please correct me if this
is
the case.

On Thu, 2018-05-31 at 17:25 -0700, Pieter Wuille via bitcoin-dev wrote:
> The best argument for why Graftroot does not need to be optional I
> think was how Greg put it: "since the signer(s) could have signed an
> arbitrary transaction instead, being able to delegate is strictly
> less
> powerful.".

I'm trying to get a more abstract view of the problem. One issue with
Greg's argument is the following:

If g-script is a script (containing a public key) that allows for
Graftroot spending, then the following "flow" of coins is valid: 

  g-script --g-sig--> script1 ---tx2---> script2

Here, g-sig is a Graftroot signature on script1 and tx2 is a
transaction that fulfills script1 and sends to script2. In other words,
the only transaction involved here is tx2; it provides g-sig, script1,
and a solution for tx1, and it spends to script2.

Now Greg's argument (as I understand it) is that this can be already
done without Grafroot with two transactions, namely a normal
transaction tx1 that spends g-script normally and tx2 that spends tx1
to script1.

  g-script ---tx1---> script1 ---tx2---> script2.

So far, so good. A difference however is that g-sig *alone* can't be
committed to the chain but tx1 alone can be committed to the chain.
That means

  g-script --g-sig--> script1 (*)

is "incomplete" but

  g-script ---tx1---> script1 (**)

is a perfectly valid transaction that can be committed to the chain. So
I think Graftroot delegation is not "strictly less powerful" than just
using a normal transaction: Graftroot enables to delegate in a way such
that the delegation itself cannot be fixed in the chain. I think this
is not possible currently. (Okay, you can just pass around the secret
keys but has other problems obviously).

Does this have practical implications?
I don't see any but maybe this helps someone to identify an undesirable
implication.

One way to be on the safe side and probably make Greg's argument go
through is to just define the semantics such that (*) is allowed, i.e.,
call g-sig a "Graftroot transaction" and give it transaction semantics.
This provides a new perspective on Graftroot: Then Graftroot does not
introduce new semantics but (*) is just an optimized version of (**)
that uses fewer bytes and may be better for privacy. 

Interestingly Andrew's blind-sig example and Johnson's fix (g-sig signs
the outpoint) are just a special case. If g-sig has transaction
semantics, it must sign the outpoint (and other stuff).

Now you can say that this is not really useful: if g-sig is essentially
a full transaction that can committed to the blockchain, then it needs
to specify inputs, outputs etc. So all the optimizations are lost and
those were the reason we want to introduce Grafroot in the first place.

But one observation here is that g-sig only needs to be a full
transaction if it's used standalone as in (*). If we want to have 

  g-script --g-sig--> script1 ---tx2---> script2

(and this should be the common case) then just the bare signature  and
script1 suffices, as in the Graftroot proposal. In some sense, inputs
and outputs of the Graftroot transaction are just implicit in this
case.

Another way to look at this that instead of providing a transaction
with g-sig, script1, and a solution for script1, you can also choose to
provide a transaction with only g-sig and script1 (and don't solve
script1), which then just sends to script1.

I'm not saying that it's worth the hassle to add this possibility
without being aware of a problem that arises if we don't add it -- but
maybe my thoughts provide another perspective on the issue.

Best,
Tim


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev