[bitcoin-dev] stratum protocol extension - mining.configure, formal version rolling and other extensions

[Jan Čapek via bitcoin-dev]

Hello,

we (at braiins systems/slushpool) would like to kindly re-open the
topic of stratum protocol extension that has been in fact implemented by
quite a few pools already (including us). This extension
currently allows configuring the stratum session for version rolling
and enables a generic mechanism for requesting protocol
extensions from the miners. More details are in the specification
below. 

I am aware of LukeJr's comments on why we haven't used
https://en.bitcoin.it/wiki/Stratum_mining_protocol#mining.capabilities_.28DRAFT.29

We are aware that there has been some academic work done on
concentrating full stratum protocol specs in one BIP as referred here
e.g.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-February/015728.html

Below is a copy of a full extension draft that has been published at
this URL:
https://github.com/slushpool/stratumprotocol/blob/master/stratum-extensions.mediawiki


We should probably join the effort and unify all the documents in one
single BIP if that would be seen as the most efficient way of having
the specs.


Kind regards,

Jan


  BIP: 
  Layer: Applications
  Title: Stratum protocol extensions
  Author: Pavel Moravec 
  Jan Capek 
  Comments-Summary: No comments yet.
  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-
  Status: Draft
  Type: Informational
  Created: 2018-03-10
  License: BSD-3-Clause
   CC0-1.0


==Abstract==

This BIP provides a generic mechanism for specifying stratum protocol
extensions. At the same time, one of the important extensions that is
specified by this BIP is configuration of bits for "version rolling"
in nVersion field of bitcoin block header.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119.

==Motivation==

The initial motivation for specifying some general support for stratum
protocol extensions was a need to allow miners to do so called
"version rolling", changing value in the first field of the Bitcoin
block header.

Version rolling is backwards incompatible change to the stratum protocol
because the miner couldn't communicate different block version value to
the server in the original version of the stratum protocol. Similarly,
a server couldn't communicate safe bits for rolling to a miner. So
both miners and pools need to implement some protocol extension to
support version rolling.

Typically, if a miner sends an unknown message to a server, the server
closes the connection (not all implementations do that but some
do). So it is not very safe to try to send unknown messages to
servers.

We can use this opportunity to make one backwards incompatible
change to the protocol to support multiple extensions in the
future. In a way that a miner can advertise its capabilities and at
the same time it can request some needed features from the server.

It is preferable that the same mechanism for feature negotiation can
be used for not yet known features. It SHOULD be easy to implement in
the mining software too.

We introduce one new message to the stratum protocol
('''"mining.configure"''') which handles the initial
configuration/negotiation of features in a generic way. So that adding
features in the future can be done without a necessity to add new
messages to stratum protocol.

Each extension has its unique string name, so called '''extension
code'''.


==Specification==
Currently, the following extensions are defined:

* '''"version-rolling"'''
* '''"minimum-difficulty"'''
* '''"subscribe-extranonce"'''


===Additional data types===

The following names are used as type aliases, making the message
description easier.

* '''TMask''' - case independent hexadecimal string of length 8,
  encoding an unsigned 32-bit integer (~[0-9a-fA-F]{8})

* '''TExtensionCode''' - non-empty string with a value equal to the
  name of some protocol extension.

* '''TExtensionResult''' - true / false /
  ''String''. ** true = The requested feature is supported
  and its configuration understood and applied. ** false =
  The feature is not supported or unknown. ** ''String'' = Error
  message containing information about what went wrong.


===Request "mining.configure"===

This message (JSON RPC Request) SHOULD be the '''first message''' sent
by the miner after the connection with the server is established. The
client uses the message to advertise its features and to request/allow
some protocol extensions.

The reason for it being the first is that we want the implementation and
possible interactions to be as easy and simple as possible. An extension
can define explicitly what does a repeated configuration of that
extension mean.

Each extension code provides a namespace for its extension parameters
and extension return values. By convention, the names are formed from
extension codes by adding "." and a 

Re: [bitcoin-dev] BIP proposal: Reserved nversion bits in blockheader - stratum mining.configure

[Jan Čapek via bitcoin-dev]

Hello,

Our reasoning for coming up with a new method for miner configuration
was stated here: https://github.com/slushpool/stratumprotocol/issues/1

It is primarily the determinism of expecting the response. That is
the reason why we chose a new method mining.configure instead of an
existing mining.capabilities that was not being very well documented or
used.


On Wed, 7 Mar 2018 14:43:11 + Luke Dashjr via bitcoin-dev
 wrote:

> Why are you posting this obsolete draft? You've already received
> review in private, and been given useful suggestions. There's even a
> shared Google Doc with the current draft:
> 
> 
> https://docs.google.com/document/d/1GedKia78NUAtylCzeRD3lMlLrpPVBFg9TV9LRqvStak/edit?usp=sharing
> 
> Again:
> 
> * This is no different from what Timo and Sergio proposed years ago,
> and as such should be based on their work instead of outright
> not-invented-here respecification. The current draft integrates their
> work while not trying to steal credit for it (they are included as
> primary authors).
> 
> * The specification should be complete, including updates for GBT and
> the Stratum mining protocol. These are included in the current draft.
> 
> Additionally, it is not appropriate to begin using a draft BIP on
> mainnet before any discussion or consensus has been reached. Doing so
> seems quite malicious, in fact. I hope DragonMint miners can still
> operate using the *current* Bitcoin protocol.
> 
> Luke
> 
> 
> On Wednesday 07 March 2018 8:19:57 AM Btc Drak via bitcoin-dev wrote:
> > Hi,
> > 
> > The following proposal reduces the number of version-bits that can
> > be used for parallel soft-fork signalling, reserving 16 bits for
> > non-specific use. This would reduce the number of parallel
> > soft-fork activations using versionbits to from 29 to 13 and
> > prevent node software from emitting false warnings about unknown
> > signalling bits under the versionbits signalling system (BIP8/9). I
> > chose the upper bits of the nVersion, because looking at the
> > versionbits implementation in the most widely deployed node
> > software, it is easier to implement than say annexing the lower 2
> > bytes of the field.
> > 
> > The scope of the BIP is deliberately limited to reserving bits for
> > general use without specifying specific uses for each bit, although
> > there have previously been various discussions of some use-cases of
> > nVersion bits including version-rolling AsicBoost[1], and nonce
> > rolling to reduce CPU load on mining controllers because
> > ntime-rolling can only be done for short periods otherwise it could
> > have negative side effects distorting time. However, specific use
> > cases are not important for this BIP.
> > 
> > I am reviving discussion on this topic now, specifically, because
> > the new DragonMint miner uses version-rolling AsicBoost on
> > mainnet[2]. It is important to bring up so node software can adapt
> > the versionbits warning system to prevent false positives. This BIP
> > has the added advantage that when a new use for bits is found,
> > mining manufacturers can play in the designated area without
> > causing disruption or inconvenience (as unfortuntely, the use of
> > version-rolling will cause until BIP8/9 warning systems are
> > adapted). I appologise for the inconvenience in advance, but this
> > is the unfortunate result of restraints while negotiating to get
> > the patent opened[3] and licensed defensively[4] in the first place.
> > 
> > I believe there was a similar proposal[5] made some years ago,
> > before the advent of BIP9. This proposal differs in that it's
> > primary purpose is to remove bits from the versionbits soft-fork
> > activation system and earmark 16 bits for general use without
> > allocating fixed uses for each bit. The BIP cites a couple of
> > usecases for good measure, but they are just informational
> > examples, not part of a specification laid down. For this reason,
> > there no is mention of the version-rolling Stratum extension[6]
> > specifics within the BIP text other than a reference to the
> > specification itself.
> > 
> > Refs:
> > 
> > [1] https://arxiv.org/pdf/1604.00575.pdf
> > [2]
> > https://halongmining.com/blog/2018/03/07/dragonmint-btc-miner-uses-version->
> > rolling-asicboost/ [3]
> > https://www.asicboost.com/single-post/2018/03/01/opening-asicboost-for-defe
> > nsive-use/ [4] https://blockchaindpl.org/ [5]
> > https://github.com/BlockheaderNonce2/bitcoin/wiki [6]
> > http://stratumprotocol.org/stratum-extensions
> > 
> > 
> >   BIP: ?
> >   Title: Reserved nversion bits in blockheader
> >   Author: BtcDrak 
> >   Comments-Summary: No comments yet.
> >   Comments-URI:
> > https://github.com/bitcoin/bips/wiki/Comments:BIP- Status: Draft
> >   Type: Informational
> >   Created: 2018-03-01
> >   License: BSD-3-Clause
> >CC0-1.0
> > 
> > 
> > ==Abstract==
> > 
> > This BIP reserves 16 bits of the block header 

Re: [bitcoin-dev] BIP Proposal: Inhibiting a covert optimization on the Bitcoin POW function

[Jan Čapek via bitcoin-dev]

Hi,

1 comment below
On Fri, 7 Apr 2017 17:52:17 -0300
Sergio Demian Lerner via bitcoin-dev
 wrote:

> 
>   BIP: TBD
>   Layer: Consensus
>   Title: Inhibiting a covert optimization on the Bitcoin POW function
>   Author: Sergio Demian Lerner 
>   Status: Draft
>   Type: Standards Track
>   Created: 2016-04-07
>   License: PD
> 
> 
> ==Abstract==
> 
> This proposal inhibits the covert use of a known optimization in
> Bitcoin Proof of Work function.
> 
> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
> "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
> document are to be interpreted as described in RFC 2119.
> 
> ==Motivation==
> 
> Due to a design oversight the Bitcoin proof of work function has a
> potential optimization which can allow a rational miner to save up-to
> 30% of their energy
> costs (though closer to 20% is more likely due to implementation
> overheads).
> 
> Timo Hanke and Sergio Demian Lerner applied for a patent on this
> optimization. The company "Sunrise Tech Group, Llc" has offered to
> license it to any interested party in the past. Sunrise Tech Group
> has been marketing their patent licenses under the trade-name
> ASICBOOST.  The document takes no position on the validity or
> enforceability of the patent.
> 
> There are two major ways of taking advantage of this optimization, as
> described
> by the patent:
> One way which is highly detectable and is not in use on the network
> today and a covert way which has significant interaction and potential
> interference with the Bitcoin protocol.  The covert mechanism is not
> easily detected except through its interference with the protocol.
> 
> In particular, the protocol interactions of the covert method can
> block the implementation of virtuous improvements such as segregated
> witness.
> 
> The use of this optimization could result in a big payoff, but the
> actual sum depends on the degree of research, investment and effort
> put into designing
> the improved cores.
> 
> On the above basis the potential for covert use of this optimization
> in the covert form and interference with useful improvements presents
> a danger to the Bitcoin system.
> 
> ==Background==
> 
> The general idea of this optimization is that SHA2-256 is a merkle
> damgard hash
> function which consumes 64 bytes of data at a time.
> 
> The Bitcoin mining process repeatedly hashes an 80-byte 'block
> header' while incriminating a 32-bit nonce which is at the end of
> this header data. This means that the processing of the header
> involves two runs of the compression function run-- one that consumes
> the first 64 bytes of the header and a second which processes the
> remaining 16 bytes and padding.
> 
> The initial 'message expansion' operations in each step of the
> SHA2-256 function operate exclusively on that step's 64-bytes of
> input with no influence from prior data that entered the hash.
> 
> Because of this if a miner is able to prepare a block header with
> multiple distinct first 64-byte chunks but identical 16-byte
> second chunks they can reuse the computation of the initial
> expansion for multiple trials. This reduces power consumption.
> 
> There are two broad ways of making use of this optimization. The
> obvious way is to try candidates with different version numbers.
> Beyond upsetting the soft-fork detection logic in Bitcoin nodes this
> has little negative effect but it is highly conspicuous and easily
> blocked.
> 
> The other method is based on the fact that the merkle root
> committing to the transactions is contained in the first 64-bytes
> except for the last 4 bytes of it.  If the miner finds multiple
> candidate root values which have the same final 32-bit then they
> can use the optimization.
> 
> To find multiple roots with the same trailing 32-bits the miner can
> use efficient collision finding mechanism which will find a match
> with as little as 2^16 candidate roots expected, 2^24 operations to
> find a 4-way hit, though low memory approaches require more
> computation.
> 
> An obvious way to generate different candidates is to grind the
> coinbase extra-nonce but for non-empty blocks each attempt will
> require 13 or so additional sha2 runs which is very inefficient.
> 
> This inefficiency can be avoided by computing a sqrt number of
> candidates of the left side of the hash tree (e.g. using extra
> nonce grinding) then an additional sqrt number of candidates of
> the right  side of the tree using transaction permutation or
> substitution of a small number of transactions.  All combinations
> of the left and right side are then combined with only a single
> hashing operation virtually eliminating all tree related
> overhead.
> 
> With this final optimization finding a 4-way collision with a
> moderate amount of memory requires ~2^24 hashing operations
> instead of the >2^28 operations that would be require for
>