Re: [bitcoin-dev] BIP process friction

[Karl-Johan Alm via bitcoin-dev]

Hello,

First off, apologies about my lack of participation. I am working on
mostly unrelated things and I'm afraid I have failed the community in
terms of what I can do on my end to keep the BIP process functional.

As such I am hereby resigning as BIP editor effective immediately.
Please remove my access privileges to the repository when possible.

-Kalle.

On Fri, 19 Jan 2024 at 09:46, Anthony Towns via bitcoin-dev
 wrote:
>
> On Thu, Jan 18, 2024 at 05:41:14AM -1000, David A. Harding via bitcoin-dev 
> wrote:
> > Question: is there a recommended way to produce a shorter identifier for
> > inline use in reading material?  For example, for proposal
> > BIN-2024-0001-000, I'm thinking:
> >
> > - BIN24-1 (references whatever the current version of the proposal is)
> > - BIN24-1.0 (references revision 0)
> >
> > I think that doesn't look too bad even if there are over 100 proposals a
> > year, with some of them getting into over a hundred revisions:
> >
> > - BIN24-123
> > - BIN24-123.123
>
> Having lived through y2k, two-digit years give me the ick, but otherwise
> sure.
>
> Cheers,
> aj, that's how the kids who didn't live through y2k say it, right?
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP extensions

[Karl-Johan Alm via bitcoin-dev]

Hi Frederico,

Welcome to the bitcoin-dev list. :)

Michael Folkson is currently pushing for a revision to BIP 2, which is
discussed in the "BIP process meeting" thread here. You could help out
by participating in that process. There's a wiki page with ideas for
this in [1] and the current plan is to modify [2] or some other pull
request to reflect what everyone decides.

[1] https://github.com/bitcoin/bips/wiki/BIP-Process-wishlist
[2] https://github.com/bitcoin/bips/pull/1015

-Kalle.

On Wed, 15 Sept 2021 at 17:29, Federico Berrone via bitcoin-dev
 wrote:
>
> Hi Karl-Johan,
> I fully agree with your proposal. In order to de-clutter BIPs and make a
> more understandable proposal, we can add the additional information in a
> separate piece. Also, this would maintain the original proposal without
> any modifications, showing the original spirit of it.
> Let me know how can I help you with your proposal.
>
> Regards,
> Federico Berrone.
>
> P/D: This is my first participation in the bitcoin-dev list, sorry if I
> am breaking any rule, I would be glad to know if that is the case.
>
> El 15/09/2021 a las 8:14, Karl-Johan Alm via bitcoin-dev escribió:
> > BIPs are proposals.
> >
> > They begin as ideas, are formulated and discussed on this list, and
> > assuming no glaring flaws are observed, turned into pull requests to
> > the bips repository, assigned a BIP number by the editors, and merged.
> >
> > It is then organically incorporated into the various entities that
> > exist in the Bitcoin space. At this point, it is not merely a
> > proposal, but a standard. As entities place their weight behind a BIP,
> > it makes less and less sense to consider its author the "maintainer"
> > of the BIP, with rights to modify it at their whim. Someone may have
> > agreed to the proposal in its original form, but they may disagree
> > with it if it is altered from under their feet.
> >
> > BIPs are modified for primarily three reasons:
> >
> > 1. Because of spelling errors, or to otherwise improve on their
> > description without changing what is actually proposed.
> > 2. To improve the proposal in some way, e.g. after discussion or after
> > getting feedback on the proposed approach.
> > 3. To add missing content, such as activation strategy.
> >
> > I propose that changes of the second and third type, unless they are
> > absolutely free from contention, are done as BIP extensions.
> >
> > BIP extensions are separate BIPs that extend on or an existing BIP.
> > BIP extensions do not require the approval of the extended-upon BIP's
> > author, and are considered independent proposals entirely. A BIP that
> > extends on BIP XXX is referred to as BIP-XXX-Y, e.g. BIP-123-1, and
> > their introductory section must include the wording "
> >
> > This BIP extends on (link: BIP-XXX).
> >
> > ".
> >
> > By making extensions to BIPs, rather than modifying them long after
> > review, we are giving the community
> > 1. the assurance that a BIP will mostly remain in its form forever,
> > except if an obvious win is discovered,
> > 2. the ability to judge modifications to BIPs, such as activation
> > parameters, on their merits alone, and
> > 3. the path to propose modifications to BIPs even if their authors
> > have gone inactive and cease to provide feedback, as is the case for
> > many BIPs today, as BIP extensions do not require the approval of the
> > extended-upon BIP.
> >
> > (Apologies if this has been proposed already. If so, feel free to
> > ignore this message, and sorry to have wasted your time.)
> > ___
> > bitcoin-dev mailing list
> > bitcoin-dev@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP extensions

[Karl-Johan Alm via bitcoin-dev]

BIPs are proposals.

They begin as ideas, are formulated and discussed on this list, and
assuming no glaring flaws are observed, turned into pull requests to
the bips repository, assigned a BIP number by the editors, and merged.

It is then organically incorporated into the various entities that
exist in the Bitcoin space. At this point, it is not merely a
proposal, but a standard. As entities place their weight behind a BIP,
it makes less and less sense to consider its author the "maintainer"
of the BIP, with rights to modify it at their whim. Someone may have
agreed to the proposal in its original form, but they may disagree
with it if it is altered from under their feet.

BIPs are modified for primarily three reasons:

1. Because of spelling errors, or to otherwise improve on their
description without changing what is actually proposed.
2. To improve the proposal in some way, e.g. after discussion or after
getting feedback on the proposed approach.
3. To add missing content, such as activation strategy.

I propose that changes of the second and third type, unless they are
absolutely free from contention, are done as BIP extensions.

BIP extensions are separate BIPs that extend on or an existing BIP.
BIP extensions do not require the approval of the extended-upon BIP's
author, and are considered independent proposals entirely. A BIP that
extends on BIP XXX is referred to as BIP-XXX-Y, e.g. BIP-123-1, and
their introductory section must include the wording "

This BIP extends on (link: BIP-XXX).

".

By making extensions to BIPs, rather than modifying them long after
review, we are giving the community
1. the assurance that a BIP will mostly remain in its form forever,
except if an obvious win is discovered,
2. the ability to judge modifications to BIPs, such as activation
parameters, on their merits alone, and
3. the path to propose modifications to BIPs even if their authors
have gone inactive and cease to provide feedback, as is the case for
many BIPs today, as BIP extensions do not require the approval of the
extended-upon BIP.

(Apologies if this has been proposed already. If so, feel free to
ignore this message, and sorry to have wasted your time.)
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] PSA: Taproot loss of quantum protections

[Karl-Johan Alm via bitcoin-dev]

On Tue, 16 Mar 2021 at 07:48, Matt Corallo via bitcoin-dev
 wrote:
>
> Overall, the tradeoffs here seem ludicrous, given that any QC issues in 
> Bitcoin need to be solved in another way, and
> can't practically be solved by just relying on the existing hash indirection.

The important distinction here is that, with hashes, an attacker has
to race against the spending transaction confirming, whereas with
naked pubkeys, the attacker doesn't have to wait for a spend to occur,
drastically increasing the available time to attack.

It may initially take months to break a single key. In such a
scenario, anyone with a hashed pubkey would be completely safe* (even
at spend time), until that speeds up significantly, while Super Secure
Exchange X with an ultra-cold 38-of-38 multisig setup using Taproot
would have a timer ticking, since the attacker need only find a single
privkey like with any old P2PK output.

(* assuming no address reuse)
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP-0322 (generic signmessage) improvements

[Karl-Johan Alm via bitcoin-dev]

Thanks a lot for taking the time to brush up the BIP. For what it's
worth, I am all for these changes, and I see them as clear
improvements all around.

IIRC Pieter was the one who originally suggested the two-checks
approach (invalid, inconclusive, valid) which is being modified here,
so would be good if you chimed in (or not -- which I'll assume means
you don't mind).

On Sat, Dec 19, 2020 at 12:27 AM Andrew Poelstra via bitcoin-dev
 wrote:
>
> I have gone over BIP-0322 and substantially rewritten the text.
> Everything I did is (I think) simply clarifying the existing
> protocol, which felt like it was written by committee and wasn't
> easy to follow, EXCEPT:
>
> 1. I rewrote the motivation section, which I believe originally
>was a paraphrase of Luke-jr's general objections to having any
>signmessage functionality. I hope Luke in particular can take
>a look at what I wrote under "Motivation" and see if it
>captures his concerns.
>
> 2. I merged the "consensus" and "upgradeable" rules to simply be
>one set of rules, consisting of consensus checks plus additional
>restrictions, all of which must be included. The new "Extensions"
>section allows validators to output the state "consensus-valid"
>if they really don't want to check the additional restrictions.
>
> 3. The "inconclusive" state, which was originally used for what I've
>called "consensus-valid", now indicates that a validator does not
>understand the script that it is checking (also described in the
>new "Extensions" section). The goal is that implementors are able
>to be meaningfully BIP-0322 while only supporting a subset of
>Script, e.g. the templates that their own software supports, or
>Miniscript, or the non-raw non-address set of output descriptors,
>or whatever.
>
>We have seen opposition to supporting BIP-322, e.g. [1] because
>of the requirement that you either have a full script interpreter
>(plus an open-ended list of Core's standardness flags, which is
>not even available through libbitcoinconsensus) or nothing. On
>the other hand, the vast majority of outputs are single-key p2pkh,
>p2pkwh or p2sh-wpkh.
>
> The new text is here (and for posterity I will also include it
> inline below, though unless Github deletes it it will be easier
> to read in rendered form):
>
> https://github.com/apoelstra/bips/blob/2020-12--bip322-overhaul/bip-0322.mediawiki
>
> I'll also PR this to the BIPs repo in the next day or two, and
> comments on Github are then welcome.
>
>
> [1] https://bitcointalk.org/index.php?topic=5261605.0
>
>
>
> * * * * * Full text of the above link * * * * *
>
> 
>   BIP: 322
>   Layer: Applications
>   Title: Generic Signed Message Format
>   Author: Karl-Johan Alm 
>   Comments-Summary: No comments yet.
>   Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322
>   Status: Draft
>   Type: Standards Track
>   Created: 2018-09-10
>   License: CC0-1.0
> 
>
> == Abstract ==
>
> A standard for interoperable signed messages based on the Bitcoin Script 
> format, either for proving fund availability, or committing to a message as 
> the intended recipient of funds sent to the invoice address.
>
> == Motivation ==
>
> The current message signing standard only works for P2PKH (1...) invoice 
> addresses. We propose to extend and generalize the standard by using a 
> Bitcoin Script based approach. This ensures that any coins, no matter what 
> script they are controlled by, can in-principle be signed for. For easy 
> interoperability with existing signing hardware, we also define a signature 
> message format which resembles a Bitcoin transaction (except that it contains 
> an invalid input, so it cannot be spent on any real network).
>
> Additionally, the current message signature format uses ECDSA signatures 
> which do not commit to the public key, meaning that they do not actually 
> prove knowledge of any secret keys. (Indeed, valid signatures can be tweaked 
> by 3rd parties to become valid signatures on certain related keys.)
>
> Ultimately no message signing protocol can actually prove control of funds, 
> both because a signature is obsolete as soon as it is created, and because 
> the possessor of a secret key may be willing to sign messages on others' 
> behalf even if it would not sign actual transactions. No signmessage protocol 
> can fix these limitations.
>
> == Types of Signatures ==
>
> This BIP specifies three formats for signing messages: ''legacy'', ''simple'' 
> and ''full''. Additionally, a variant of the ''full'' format can be used to 
> demonstrate control over a set of UTXOs.
>
> === Legacy ===
>
> New proofs should use the new format for all invoice address formats, 
> including P2PKH.
>
> The legacy format MAY be used, but must be restricted to the legacy P2PKH 
> invoice address format.
>
> === Simple ===
>
> A ''simple'' signature consists of a witness stack, consensus encoded as a 
> vector of vectors of 

[bitcoin-dev] Message signing (again)

[Karl-Johan Alm via bitcoin-dev]

Hello,

I have updated the signmessage proposal (BIP-322) to use the same
approach as signet uses, which brings out of the box support for psbt
and such, and removes the need for a custom signer and validator
(well, sort of anyway).

In the process, I've also replaced the concatenation approach
(hash("Bitcoin Signed Message || ")) with a
BIP340-tagged-hash approach (h/t @ajtowns).

Not much remains of the old BIP, so I am tentatively submitting this
as a replacement proposal. I'd be totally fine with submitting this as
an updated BIP-322 though, if people prefer that.

Pull request is here:

https://github.com/bitcoin/bips/pull/1003

Viewable version:

https://github.com/bitcoin/bips/blob/ce60832ef41301105a95c15dcd854d8ecbc53e00/bip-signmessage-redone.mediawiki

Inline version:


BIP: 
Layer: Applications
Title: Generic Signed Message Format
Author: Karl-Johan Alm 
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-
Status: Draft
Type: Standards Track
Created: 2020-10-01
License: CC0-1.0
Replaces: 322


== Abstract ==

A standard for interoperable generic signed messages based on the
Bitcoin Script format.

== Background ==

* Assume two actors, a prover P and a verifier V.
* P wants to prove that they own the private key
k associated with a given address A (which
in turn is derived from the pubkey kG).
* Let V generate a message M and hand this
to P.
* P generates a signature S by signing the
message M using k. Given S,
V can prove that P has the private key
associated with A.

The astute reader will notice that the above is missing a critical
part, namely the pubkey kG, without which the verifier
cannot actually verify the message. The current message signing
standard solves this via a cryptographic trick, wherein the signature
S above is a special "recoverable signature" type. Given
the message M and the signature S, it is
then possible to recover the pubkey kG. The system thus
derives the address for the pubkey kG, and if it does not
match A, the proof is deemed invalid.

While this is a neat trick, it unnecessarily restricts and complicates
the message signing mechanism; for instance, it is currently not
possible to sign a message for a P2SH address, because there is no
pubkey to recover from the resulting signature.

== Motivation ==

The current message signing standard only works for P2PKH (1...)
addresses. By extending it to use a Bitcoin Script based approach, it
could be made more generic without causing a too big burden on
implementers, who most likely have access to Bitcoin Script
interpreters already.

== Specification ==

This BIP follows the specification of BIP-325 challenges and solutions.

Let there be two virtual transactions to_spend and to_sign.

The "to_spend" transaction is:

nVersion = 0
nLockTime = 0
vin[0].prevout.hash = ...000
vin[0].prevout.n = 0x
vin[0].nSequence = 0
vin[0].scriptSig = OP_0 PUSH32[ message_hash ]
vin[0].scriptWitness = []
vout[0].nValue = 0
vout[0].scriptPubKey = message_challenge

where message_hash is a BIP340-tagged hash of the message, i.e.
sha256_tag(m), where tag = "BIP-signed-message", and
message_challenge is the to be proven (public) key script.

The "to_sign" transaction is:

nVersion = 0
nLockTime = 0
vin[0].prevout.hash = to_spend.txid
vin[0].prevout.n = 0
vin[0].nSequence = 0
vout[0].nValue = 0
vout[0].scriptPubKey = message_signature

It may include other inputs, to facilitate a proof of funds.

A message signature is considered valid, inconclusive, or invalid
based on whether the to_sign transaction is a valid spend of the
to_spend transaction or not, according to the following steps (also
see Consensus and standard flags section further down):

1. Valid, if it is a successful spend according to the current
consensus rules (sometimes called "policy").
2. Inconclusive, if it is a successful spend according to consensus
rules, but NOT according to policy rules
3. Invalid, if it is not a successful spend according to consensus rules

A proof is the base64-encoding of the message_signature as is. A
validator takes the to be proven pubkey and the proof and transforms
it to virtual transactions as described above.

== Legacy format ==

The legacy format is restricted to the legacy P2PKH address format.

Any other input (i.e. non-P2PKH address format) must be signed using
the new format described above.

=== Signing ===

Given the P2PKH address a and the message m,
and the pubkey-hash function pkh(P) =
ripemd160(sha256(P)):

# let p be the pubkey-hash pkh(P) for the
pubkey P, contained in a
# let x be the private key associated with P
so that pkh(xG) = p
# let digest be SHA56d("Bitcoin Signed Message:\n"||m)
# create a compact signature sig (aka "recoverable ECDSA
signature") using x on digest

The resulting proof is sig, serialized using the base64 encoding.

=== Verifying ===

Given the P2PKH address a, the message m,
the compact signature sig, and the pubkey-hash function

Re: [bitcoin-dev] RFC: Kicking BIP-322 (message signing) into motion

[Karl-Johan Alm via bitcoin-dev]

220119c2995c1661138142f3899c1fd6d1af7e790e0e081be72db9c
e7bf5b5b932901210290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e7c1dee9e
a6e2dcd869

Base64: AQEAAkcwRAIgdbT7QEIdVcVUYoecs1KoXus68hONPwKQLJFD8Shw9fcCIBGcKZXBZhE4
FC84mcH9bRr355Dg4IG+ctuc579bW5MpASECkL7M0Ctz7KV0Z7K28eRxYam3al5nWG58He6e
puLc2Gk=


Split into components:

{|class="wikitable" style="text-align: center;"
|-
!Type
!Length
!Name
!Value
!Comment
|-
|Uint32||4||flags||0100||proof format version
|-
|Uint8||1||entries||01||1 entry
|-
|VarInt||1-8||scriptsiglen||00||0 byte scriptsig
|-
|VarInt||1-8||wit entries||02||2 witness stack entries
|-
|VarInt||1-8||entry1len||47||71 byte entry
|-
|Uint8[71]||71||entry1||3044022075b4fb40421d55c55462879cb352a85eeb3af213
8d3f02902c9143f12870f5f70220119c2995c1661138142f
3899c1fd6d1af7e790e0e081be72db9ce7bf5b5b932901||Witness stack item 1
|-
|VarInt||1-8||entry2len||21||33 byte entry
|-
|Uint8[33]||33||entry2||0290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e
7c1dee9ea6e2dcd869||Witness stack item 2
|}

The above test vector is for a bech32 P2WPKH (native segwit) address.
(Once BIP solidifies, will add test vector for other types.)
```

On Wed, Mar 4, 2020 at 11:43 PM Greg Sanders  wrote:
>
> OP_MESSAGEONLY would make "dumb" signers like HWW more difficult to support. 
> They'd have to do script interpretation to make sure they're not signing 
> something real with funds.
>
> Just FYI.
>
> On Wed, Mar 4, 2020 at 9:35 AM Luke Dashjr via bitcoin-dev 
>  wrote:
>>
>> In addition to starting with proof-of-funds instead of proof-of-receiver, it
>> would be nice to integrate with Taproot somehow or another. Perhaps
>> OP_MESSAGEONLY is the most straightforward way to do this? It might be a good
>> idea to have a message type after the opcode too.
>>
>> On Wednesday 04 March 2020 06:23:53 Karl-Johan Alm via bitcoin-dev wrote:
>> > Hello,
>> >
>> > I noticed recently that a PR to Bitcoin Core that pretty much touched
>> > everything my BIP-322 pull request touches (around the same
>> > complexity) was merged without a thought given to BIP-322
>> > compatibility, despite the BIP-322 PR being open for 2x the time. I
>> > can only conclude from this that people dislike BIP-322 in its current
>> > form, which the 9 month old pull request stagnating can probably
>> > attest to.
>> >
>> > There are several things that I can do to make this a bit more
>> > appealing to people, which would hopefully kick the progress on this
>> > forward. I have already put in a non-trivial amount of energy and
>> > effort into maintaining the pull request as is, so I'd prefer if
>> > people were harsh and unfiltered in their criticism rather than polite
>> > and buffered, so I can beat this thing into shape (or abandon it, in
>> > the worst case).
>> >
>> > =
>> > 1. People use signmessage as a way to prove funds. This is misleading
>> > and should be discouraged; throw the sign message stuff out and
>> > replace it entirely with a prove funds system.
>> >
>> > I know in particular luke-jr is of this opinion, and Greg Maxwell in
>> > https://github.com/bitcoin/bitcoin/pull/16440#issuecomment-568194168
>> > leans towards this opinion as well, it seems.
>> >
>> > =
>> > 2. Use a transaction rather than a new format; make the first input's
>> > txid the message hash to ensure the tx cannot be broadcasted. This has
>> > the benefit of being able to provide to an existing hardware wallet
>> > without making any modifications to its firmware.
>> >
>> > I think Mark Friedenbach and Johnson Lau are of this opinion, except
>> > Johnson Lau also suggests that the signature hash is modified, see
>> > https://github.com/bitcoin/bips/pull/725#issuecomment-420040430 --
>> > which defeats the benefit above since now hw wallets can no longer
>> > sign.
>> >
>> > Prusnak (I think he works at Trezor; apologies if I am mistaken) is
>> > against this idea, and proposes (3) below:
>> > https://github.com/bitcoin/bips/pull/725#issuecomment-420210488
>> >
>> > =
>> > 3. Use Trezor style
>> >
>> > See https://github.com/trezor/trezor-mcu/issues/169
>> >
>> > This has the benefit of already being adopted (which clearly BIP-322
>> > is failing hard at right now), but has the drawback that we can no
>> > longer do *generic* signing; we are stuck with the exact same
>> > limitations as in the legacy system, which we kinda wanted to fix in
>> > the updated version.
>> >
>> &

[bitcoin-dev] Signet: static genesis block, and dynamic message start

[Karl-Johan Alm via bitcoin-dev]

Hello,

I am proposing a modification to BIP-325 to make the genesis block
static and to rely on the message start to avoid collision between
signets when multiple nets exist simultaneously:

https://github.com/bitcoin/bips/pull/900
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] New tool to assist in BIP 340-342 review: tap

[Karl-Johan Alm via bitcoin-dev]

Hello,

One of the tools I am maintaining called btcdeb (Bitcoin Script
Debugger) has a new experimental branch "taproot", which builds on top
of the WIP taproot pull request to Bitcoin Core, and contains a new
command line tool called "tap".

Tap allows you to compose taproot and/or tapscript outputs, both for
funding and spending.

There is a detailed tutorial on how this works here:
https://github.com/kallewoof/btcdeb/blob/taproot/doc/tapscript-example-with-tap.md

Feedback welcome.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Is Signet Bitcoin?

[Karl-Johan Alm via bitcoin-dev]

Hello,

The pull request to the bips repository for Signet has stalled, as the
maintainer isn't sure Signet should have a BIP at all, i.e. "is Signet
Bitcoin?".

My argument is that Signet is indeed Bitcoin and should have a BIP, as
this facilitates the interoperability between different software in
the Bitcoin space.

Feedback welcome, here or on the pull request itself:
https://github.com/bitcoin/bips/pull/803
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIPable-idea: Consistent and better definition of the term 'address'

[Karl-Johan Alm via bitcoin-dev]

I've proposed bitcoin invoice for awhile now. See
https://twitter.com/kallewoof/status/1165841566105079808

I like bitcoin invoice address into bitcoin address as proposed by Chris.


On Fri, Oct 11, 2019 at 12:45 AM Emil Engler via bitcoin-dev
 wrote:
>
> * Sorry if this mail was sent multiple times, my E-Mail client went crazy *
>
> Thanks for all your feedback.
> I came to the decision to write a BIP for this, even if it might not be
> implemented by many wallets, a standardization is never wrong and this
> would be the first step in the correct direction for better on-chain
> privacy.
>
> However currently we still need a good term for the 'address' replacement.
>
> The current suggestions are:
> * Invoice ID
> * Payment Token
> * Bitcoin invoice (address)
> * Bitcoin invoice (path)
>
> Because of the LN term invoice I really like the term 'Bitcoin Invoice'
> by Chris Belcher.
>
> So how do find a consensus about these terms?
>
> Greetings
> Emil Engler
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] 32-byte public keys in Schnorr and Taproot

[Karl-Johan Alm via bitcoin-dev]

Hello,

It makes no sense to me to not switch to 32-byte keys. There are
literally no (or very mild) disadvantages to this, from what it
appears like. I don't think refraining from updating a proposal just
because it's been out there for awhile is a valid reason, personally.

On Sat, Aug 10, 2019 at 3:17 AM Pieter Wuille via bitcoin-dev
 wrote:
>
> Hello all,
>
> It has been suggested [1] to drop the Y oddness bit in the witness
> program for Taproot outputs. This seems like a worthwhile change, as:
> * The bit doesn't actually contribute to security.
> * It avoids Taproot outputs from being more expensive to create than v0 P2WSH.
> * It doesn't preclude future changes that would still need the
> additional byte anyway.
>
> In exploring that option, Jonas Nick found that it seems cleanest [2]
> to actually introduce a type of 32-byte public keys (which implicitly
> have an even Y coordinate) in bip-schnorr, with associated signing and
> verification logic that are distinct from the 33-byte variant.
>
> This makes me wonder if we need 33-byte public keys at all.
>
> So I'd like to hear opinions about modifying bip-schnorr to only
> define 32-byte public keys. The implications of that would be:
> * bip-schnorr public keys wouldn't be exactly the same as ECDSA public
> keys, however all derivation logic would still apply (BIP32,
> mnemonics, xpubs, ... would still exist and be compatible - just the
> first pubkey byte would be dropped at the end).
> * The Q point in bip-taproot (the one going in the scriptPubKey) would
> just follow the 32-byte pubkey encoding, rather than needing a 33rd
> byte.
> * The P point in bip-taproot (the internal key revealed during script
> path) would become just a 32-byte public key (and we can drop the one
> bit in the control block to transmit the oddness of the Y coordinate
> of P).
> * In order to permit batch verification of the P to Q tweaking for
> script-path spending, another control block bit is now needed, namely
> one that indicates whether a negation was needed to make P+H(P||m)*G's
> Y coordinate even.
> * All public keys inside bip-tapscript would also become 32-bytes. If
> desired, the "upgradable public key" construction in bip-tapscript can
> be kept, by triggering based on the length of public keys (rather than
> based on their first byte).
>
> One question is whether it's worth such a change to bip-schnorr at
> this point. We've purposefully never progressed it past draft since
> publishing [3], but it has been a while. If necessary, it's possible
> to keep verification compatible by still hashing the implied "even"
> byte inside the scheme (which commits to the pubkey), but if we're
> going to change things, it's perhaps best to do it as cleanly as
> possible and also drop that byte.
>
> Opinions?
>
>   [1] 
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-May/016943.html
>   [2] https://github.com/sipa/bips/pull/52
>   [3] 
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-July/016203.html
>
> Cheers,
>
> --
> Pieter
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Absent authors and BIP updates

[Karl-Johan Alm via bitcoin-dev]

Hello,

People come in as Bitcoin developers all the time, but sometimes
people also leave permanently. In the case of BIP authors, when a user
leaves and does not respond to (reasonable) requests to change their
BIPs, we are sort of stuck right now.

BIP-2 states that anyone is allowed to request a change of a
draft/proposed BIP to "rejected" status after 3 years of inactivity,
but sometimes BIPs are accepted, after which the author leaves.

See e.g. https://github.com/bitcoin/bips/pull/339

There are a couple of ways to address this:

1. We allow anyone to request a change of a draft/proposed BIP to
"accepted", at any time. It is upon the requester to prove within
reason that the BIP does fulfill the criteria as given in BIP-2. Such
a request does not need the approval of the BIP author.

2. We allow BIPs to expire after 3 years, and allow the BIP repository
maintainer to assign a new champion, once/if such a champion appeared.

3. We do neither, and when such is warranted, we make new BIPs with
new assignations which supercede the old BIP, sort of like how BIP 2
supercedes BIP 1. Without the author to sign off on the change though,
it may not be obvious enough since we can't modify the original BIP.

4. Other approach / fine as it is.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP: Signet

[Karl-Johan Alm via bitcoin-dev]

Hello,

I have written a BIP describing the Signet network. Feedback requested!

https://github.com/bitcoin/bips/pull/803

Pasted in its entirety below, with formatting issues left as is. See
above link for styled version.


BIP: 
Layer: Applications
Title: Signet
Author: Karl-Johan Alm 
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-
Status: Draft
Type: Standards Track
Created: 2019-03-20
License: CC0-1.0


== Abstract ==

A new type of test network where signatures are used in addition to
proof of work for block progress, enabling much better coordination
and robustness (be reliably unreliable), for persistent, longer-term
testing scenarios involving multiple independent parties.

== Motivation ==

Testnet is a great place to try out new things without risking real
money, but it is notoriously unreliable. Huge block reorgs, long gaps
in between blocks being mined or sudden bursts of blocks in rapid
succession mean that realistic testing of software, especially
involving multiple independent parties running software over an
extended period of time, becomes infeasible in practice.

A new type of test network would be more suitable for integration
testing by organizations such as exchanges, or testing of next
generation Layer-2 protocols like Eltoo or sidechain pegs. The goal is
not to be perfectly reliable but rather to have a predictable amount
of unreliability. You want a test network to behave like mainnet (i.e.
no thousands of block reorgs) while also making it easier to trigger
expected but rare events like a 6-block reorg. Regtest is not suitable
for longer-term scenarios involving multiple independent parties
because creating blocks costs nothing, so any party can completely
control the test network.


== Specification ==

A new type of network ("signet"), which takes an additional consensus
parameter called the challenge (scriptPubKey). The challenge can be a
simple pubkey (P2PKH style), or a k-of-n multisig, or any other script
you would want.

The witness commitment of the coinbase transaction is extended to
include a secondary commitment (the signature/solution):

1-4 bytes - Push the following (x + 4) bytes
4 bytes - Signet header (0xecc7daa2)
x bytes - Solution (sigScript)

Any push operations that do not start with the 4 byte signet header
are ignored. Multiple push operations with the 4 byte signet header
are ignored except for the first entry.

Any signature operations contained within the challenge use
SHA256d(modifiedBlockHash), i.e. the double-SHA256 digest of the
following data as the sighash:

{|class="wikitable" style="text-align: center;"
|-
!Type
!Size
!Name
|-
|Int32||4||nVersion
|-
|Uint256||32||hashPrevBlock
|-
|Uint256||32||modifiedMerkleRoot
|-
|Uint32||4||nTime
|-
|Uint32||4||nBits
|}

The modifiedMerkleRoot hash is obtained by generating the
merkle root of the block transactions, with the coinbase witness
commitment as is, without the signet extension. This means the merkle
root of the block is different from the merkle root in the signet
commitment, but in return, the block nonce value is the only component
that the signet signature does not commit to. When grinding proof of
work, the extended nonce cannot be used as it would invalidate the
signature. Instead, simply resigning the same (or an updated) block
will give a new search space.

A block is considered fully validated if the above commitment is
found, and its solution is valid. This verification should be done
directly before or after the witness commitment verification.

== Compatibility ==

This specification is backwards compatible in the sense that existing
software can use Signet out of the box.

Simply by adding the network parameters for signet (magic number,
etc), a client can connect to and use any signet network without
further modifications. The block headers have valid proof of work, so
clients can trivially check that blocks are "probably" valid.

However, anyone can mine blocks that are accepted by the client for
any given signet network. These blocks do not contain the required
signatures, however, so any fully validating node will promptly reject
them. As such, clients need to either validate the block signature
inside the coinbase transaction, or connect to trusted peers.

Other software need not add block signature validation code that they
will not use in production. This is adequate for non-production test
purposes where the goal is to have a network behave as much like
mainnet as possible.

== Reference implementation ==

WIP implementation at https://github.com/kallewoof/bitcoin/pull/4

== Acknowledgements ==

TODO

== References ==

# Original mailing list thread:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-March/016734.html
# Bitcoin Wiki entry: https://en.bitcoin.it/wiki/Signet

== Copyright ==

This document is licensed under the Creative Commons CC0 1.0 Universal license.

Re: [bitcoin-dev] Signet

[Karl-Johan Alm via bitcoin-dev]

Hi Varunram,

On Wed, Mar 13, 2019 at 3:41 PM Varunram Ganesh
 wrote:
>
> I like your idea of a signet as it would greatly help test reorgs and stuff 
> without having to experiment with regtest. But I'm a bit concerned about 
> running a common signet (Signet1) controlled by a trusted entity. I guess if 
> someone wants to test signet on a global scale, they could spin up a couple 
> nodes in a couple places (and since it is anyway trusted, they can choose to 
> run it on centralised services like AWS). Another concern is that the 
> maintainer might have unscheduled work, emergencies, etc and that could 
> affect how people test stuff on. This would also mean that we need people to 
> run signet1 nodes in parallel with current testnet nodes (one could argue 
> that Signet is trusted anyway and this doesn't matter, still)
>
> I'm sure you would have considered these while designing, so would be great 
> to hear your thoughts.

For starters, I assume that the signer would run an automated script
that generated blocks on regular intervals without requiring manual
interaction. So even if the signer went on a vacation, the network
would keep on ticking. I also assume the signer would be running a
faucet service so users could get coins as needed. Ultimately though,
if a signer ended up vanishing or being unreliable, people would just
set up a new signet with a different signer and use that instead, so
ultimately it's not a big deal.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Signet

[Karl-Johan Alm via bitcoin-dev]

Hi Anthony,

On Wed, Mar 13, 2019 at 12:23 PM Anthony Towns  wrote:
>
> Maybe make the signature be an optional addition to the header, so
> that you can have a "light node" that doesn't download/verify sigs
> and a full node that does? (So signatures just sign the traditional
> 80-byte header, and aren't included in the block's tx merkle root, and
> the prevHash reflects the hash of the previous block's 80-byte header,
> without the signature)

This seems to be what everyone around me thinks is the best approach.
I.e. signet is a "p2p level agreement" that an additional signature is
required for a block to be considered fully valid.

It has the added complexity that a signature-aware signet node talking
to a non-signature-aware signet node should reject/discard headers
sent from the peer, or you will run into situations where a node
doesn't know if the headers are valid or not and has to hold onto them
indefinitely, which is a situation that currently does not occur in
"regular" nets.

If you detach the signature from the header, you also end up with
cases where a malicious user could send garbage data as the signature
for a valid header, forcing peers to mark that header as invalid, even
though it isn't. That is regardless of whether a fix is in place for
the above, too.

> If you did this, it might be a good idea to enforce including the previous
> block's header signature in the current block's coinbase.

Yeah that is one of the ideas we had early on, and I think it's a good
one to do. It doesn't mean someone cannot spam a bunch of invalid
headers at block height current_tip+1, but at least you can get all
but the latest signature now. So as long as you are able to fetch the
latest signature, you should theoretically be able to verify the
entire chain just from the blocks + that one sig.

-Kalle.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Signet

[Karl-Johan Alm via bitcoin-dev]

Hello all,

I started writing code that puts the signature in the coinbase
transaction similar to the witness commitment, and encountered a
potential issue. See inline comments below.

On Mon, Mar 11, 2019 at 2:02 AM David A. Harding  wrote:
>
> On Sun, Mar 10, 2019 at 09:43:43AM +0900, Karl-Johan Alm via bitcoin-dev 
> wrote:
> > Keeping the PoW rule and moving the signature would mean DoS attacks
> > would be trivial as anyone could mine blocks without a signature in
> > them
>
> Sure, but anyone could also just connect their lite client to a trusted
> node (or nodes) on signet.  The nodes would protect the clients from
> missing/invalid-signature DoS and the clients wouldn't have to implement
> any more network-level changes than they need to now for testnet.
>
> For people who don't want to run their own trusted signet nodes, there
> could be a list of signet nodes run by well-known Bitcoiners (and this
> could even be made available via a simple static dns seeder lite clients
> could use).

This sounds sensible. One issue here is that the "proper" signer will
be orders of magnitude slower than the fake miner when constructing
blocks. Because the signature is now stuffed into the coinbase
transaction, it becomes a part of the block merkle root, so the true
miner now has to (1) create a block, (2) sign it, (3) check if hash <
target, (4) nudge nonce if not, and then repeat from step (2) until it
finds a valid block. I.e. it has to sign the entire thing for every
nonce.

> This post from Maxwell could be the idea Corallo is describing:
>
> 
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-August/016348.html

That's pretty cool. The plan I had was to set up some simple interface
where anyone could "order" reorgs whenever they wanted to. It would
reorg/double spend people on request (e.g. "send 1 signetcoin to
signet1qfoobar and then double spend it in a reorg 3 blocks deep") and
so on.

With that kind of tool, I don't know if you need the alternate signing
approach you described, but I could be mistaken.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Signet

[Karl-Johan Alm via bitcoin-dev]

Hi Lautaro,

Using regtest is not ideal for public networks, as anyone anywhere can
just rewrite the blockchain at their whim by mining a ton of blocks.

On Sun, Mar 10, 2019 at 4:52 AM Lautaro Dragan  wrote:
>
> Hi Karl-Johan, my two cents:
>
> At Po.et we use regtest to simulate reorgs in integration tests in Travis / 
> CircleCI. It has proved quite useful.
>
> In general regtest for automated testing has given us all we needed so far, 
> but I admit we have a rather simple use of Bitcoin right now (colored coins).
>
> For local development, we sometimes use a script that "mines" blocks in 
> regtest periodically. It was my goal to also use this method in QA, although 
> we wound up using testnet and didn't encounter any problems so far.
>
> Out of curiosity: what limitations did you find in using, for example, a 
> private network of bitcoin core nodes running regtest? (this gives you the 
> same power as centralization without any changes or extra functionality 
> required)
>
> El vie., 8 de mar. de 2019 a la(s) 16:02, Karl-Johan Alm via bitcoin-dev 
>  escribió:
>>
>> Hello,
>>
>> As some of you already know, I've been working on a network called "signet", 
>> which is bascially a complement to the already existing testnet, except it 
>> is completely centralized, and blocks are signed by a specific key rather 
>> than using proof of work.
>>
>> Benefits of this:
>>
>> 1. It is more predictable than testnet. Miners appear and disappear 
>> regularly, causing irregular block generation.
>>
>> 2. Since it is centrally controlled, it is easy to perform global testing, 
>> such as reorgs (e.g. the network performs a 4 block reorg by request, or as 
>> scheduled).
>>
>> 3. It is more stable than testnet, which occasionally sees several thousand 
>> block reorgs.
>>
>> 4. It is trivial to spin up (and shut down) new signets to make public tests 
>> where anyone can participate.
>>
>> Anyone can create a signet at any time, simply by creating a key pair and 
>> creating a challenge (scriptPubKey). The network can then be used globally 
>> by anyone, assuming the creator sends some coins to the other participants.
>>
>> Having a persistent signet would be beneficial in particular to services 
>> which need a stable place to test features over an extended period of time. 
>> My own company implements protocols on top of Bitcoin with sidechains. We 
>> need multi-node test frameworks to behave in a predictable manner (unlike 
>> testnet) and with the same standardness relay policy as mainnet.
>>
>> Signets consist of 2 parameters: the challenge script (scriptPubKey) and the 
>> solution length. (The latter is needed to retain fixed length block headers, 
>> despite having an additional payload.)
>>
>> I propose that a default persistent "signet1" is created, which can be 
>> replaced in future versions e.g. if the coins are unwisely used as real 
>> money, similarly to what happened to previous testnets. This signet is 
>> picked by default if a user includes -signet without providing any of the 
>> parameters mentioned above. The key holder would be someone sufficiently 
>> trusted in the community, who would be willing to run the system (block 
>> generation code, faucet, etc). It could be made a little more sturdy by 
>> using 1-of-N multisig as the challenge, in case 1 <= x < N of the signers 
>> disappear. If people oppose this, it can be skipped, but will mean people 
>> can't just jump onto signet without first tracking down parameters from 
>> somewhere.
>>
>> Implementation-wise, the code adds an std::map with block hash to block 
>> signature. This is serialized/deserialized as appropriate (Segwit witness 
>> style), which means block headers in p2p messages are (80 + solution_length) 
>> bytes. Block header non-contextual check goes from checking if block header 
>> hash < target to checking if the payload is a valid signature for the block 
>> header hash instead.
>>
>> Single commit with code (will split into commits and make PR later, but just 
>> to give an idea what it looks like): 
>> https://github.com/kallewoof/bitcoin/pull/4
>>
>> I don't think this PR is overly intrusive, and I'm hoping to be able to get 
>> signet code into Bitcoin Core eventually, and am equally hopeful that devs 
>> of other (wallet etc) implementations will consider supporting it.
>>
>> Feedback requested on this.
>>
>> Attribution: parts of the signet code (in particular signblock and 
>> getnewblockhex) were adapted from the ElementsProject/elements repository. 
>> When PR is split into atomic commits, I will put appropriate attribution 
>> there.
>>
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Signet

[Karl-Johan Alm via bitcoin-dev]

Hi Matt,

On Sat, Mar 9, 2019 at 5:20 AM Matt Corallo  wrote:
>
> To make testing easier, it may make sense to keep the existing block header 
> format (and PoW) and instead apply the signature rules to some field in the 
> coinbase transaction. This means SPV clients (assuming they only connect to 
> honest/trusted nodes) work as-is.

Keeping the PoW rule and moving the signature would mean DoS attacks
would be trivial as anyone could mine blocks without a signature in
them, unless you ramped up the difficulty, which would mean it's just
another testnet. It's a test network, admittedly, but I think it would
kind of defeat the purpose.

> A previous idea regarding reorgs (that I believe Greg came up with) is to 
> allow multiple keys to sign blocks, with one signing no reorgs and one 
> signing a reorg every few blocks, allowing users to choose the behavior they 
> want.

Not sure how this would work in practice. The idea with signet is to
have an actual network that is occasionally reorged, i.e. it's a
global network (for those participating) that everyone agrees on. Not
sure how you would have choices there.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Signet

[Karl-Johan Alm via bitcoin-dev]

Hello,

As some of you already know, I've been working on a network called
"signet", which is bascially a complement to the already existing testnet,
except it is completely centralized, and blocks are signed by a specific
key rather than using proof of work.

Benefits of this:

1. It is more predictable than testnet. Miners appear and disappear
regularly, causing irregular block generation.

2. Since it is centrally controlled, it is easy to perform global testing,
such as reorgs (e.g. the network performs a 4 block reorg by request, or as
scheduled).

3. It is more stable than testnet, which occasionally sees several thousand
block reorgs.

4. It is trivial to spin up (and shut down) new signets to make public
tests where anyone can participate.

Anyone can create a signet at any time, simply by creating a key pair and
creating a challenge (scriptPubKey). The network can then be used globally
by anyone, assuming the creator sends some coins to the other participants.

Having a persistent signet would be beneficial in particular to services
which need a stable place to test features over an extended period of time.
My own company implements protocols on top of Bitcoin with sidechains. We
need multi-node test frameworks to behave in a predictable manner (unlike
testnet) and with the same standardness relay policy as mainnet.

Signets consist of 2 parameters: the challenge script (scriptPubKey) and
the solution length. (The latter is needed to retain fixed length block
headers, despite having an additional payload.)

I propose that a default persistent "signet1" is created, which can be
replaced in future versions e.g. if the coins are unwisely used as real
money, similarly to what happened to previous testnets. This signet is
picked by default if a user includes -signet without providing any of the
parameters mentioned above. The key holder would be someone sufficiently
trusted in the community, who would be willing to run the system (block
generation code, faucet, etc). It could be made a little more sturdy by
using 1-of-N multisig as the challenge, in case 1 <= x < N of the signers
disappear. If people oppose this, it can be skipped, but will mean people
can't just jump onto signet without first tracking down parameters from
somewhere.

Implementation-wise, the code adds an std::map with block hash to block
signature. This is serialized/deserialized as appropriate (Segwit witness
style), which means block headers in p2p messages are (80 +
solution_length) bytes. Block header non-contextual check goes from
checking if block header hash < target to checking if the payload is a
valid signature for the block header hash instead.

Single commit with code (will split into commits and make PR later, but
just to give an idea what it looks like):
https://github.com/kallewoof/bitcoin/pull/4

I don't think this PR is overly intrusive, and I'm hoping to be able to get
signet code into Bitcoin Core eventually, and am equally hopeful that devs
of other (wallet etc) implementations will consider supporting it.

Feedback requested on this.

Attribution: parts of the signet code (in particular signblock and
getnewblockhex) were adapted from the ElementsProject/elements repository.
When PR is split into atomic commits, I will put appropriate attribution
there.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] RFC: BIP 322: Generic Signed Message Format

[Karl-Johan Alm via bitcoin-dev]

Greetings,

(The quoted proposal is already outdated, and I recommend you check
out the up to date formatted version here:
https://github.com/kallewoof/bips/blob/bip-generic-signmessage/bip-0322.mediawiki
The PR with comments is here: https://github.com/bitcoin/bips/pull/725)

A big part of the feedback boils down to conflicts of opinion related
to whether or not proofs should be given as transactions or not.

I am attempting to write down the pros and cons below, some of which
are contradictory/complementary based on multiple people's opinions.

Pros of using transaction format:
1. works out of the box with existing HSM:s, some of which may be
purposefully built to not be upgradable for security reasons (unless
sighash modified; see below)
2. easier to incorporate into existing software
3. forward compatible with bitcoin extensions (e.g. mimblewimble,
confidential transactions, etc)
4. HSM:s *should* be blind to whether or not a transaction or a
message or proof of funds is being signed (see Con #3)

Cons:
1. dangerous if challenger is able to convince prover to sign a
message that corresponds to an actual transaction; modifying sighash
is suggested, but defeats pro #1 above; can define in tx to have txin
as sighash to guarantee invalid on chain
2. unupgraded software are unable to make distinction between message
sign and transaction sign
3. if HSM:s do not support it and do not support upgrading, this is by
design, and message signing should be explicitly supported or not be
possible (see Pro #4)
4. severely cripples UX for hardware wallets that actually show the
contents of the transaction during the signing

If anyone has comments on this, it would be tremendously appreciated.

There is also a divided opinion on whether an "OP_MESSAGEONLY" opcode
should be introduced, but I'd like to address the above first, if
possible.
On Tue, Sep 11, 2018 at 1:41 PM Karl-Johan Alm
 wrote:
>
> Hi.
>
> [note: BIP number was assigned to PR before this email was sent; I did
> not self-assign the BIP number]
>
> Below is a proposal to extend the existing sign/verifymessage format
> to a more generalized variant relying on the script verification
> mechanism in Bitcoin itself for message signing/verification, based on
> the original discussion
> (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-March/015818.html)
> .
>
> PR is here: https://github.com/bitcoin/bips/pull/725
>
> A formatted version of this text can be seen here:
> https://github.com/kallewoof/bips/blob/bip-generic-signmessage/bip-generic-signmessage.mediawiki
>
> Note: I am not sure how to best deal with CLTV/CSV stuff here, ultimately.
>
> Note 2: I have received suggestions from several people to use a
> Bitcoin transaction instead. If someone could explain why this is
> beneficial, it would be very helpful. I'm not against it, just feels
> like the whole transaction part is unnecessary complexity/overhead.
>
> ---
> 
>   BIP: 322
>   Layer: Applications
>   Title: Generic Signed Message Format
>   Author: Karl-Johan Alm 
>   Comments-Summary: No comments yet.
>   Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322
>   Status: Draft
>   Type: Standards Track
>   Created: 2018-09-10
>   License: CC0-1.0
> 
>
> == Abstract ==
>
> A standard for interoperable generic signed messages based on the
> Bitcoin Script format.
>
> == Motivation ==
>
> The current message signing standard only works for P2PKH (1...)
> addresses. By extending it to use a Bitcoin Script based approach, it
> could be made more generic without causing a too big burden on
> implementers, who most likely have access to Bitcoin Script
> interpreters already.
>
> == Specification ==
>
> A new structure SignatureProof is added, which is a
> simple serializable scriptSig & witnessProgram container.
>
> Two actions "Sign" and "Verify" are defined.
>
> === SignatureProof container ===
>
> {|class="wikitable" style="text-align: center;"
> |-
> !Type
> !Length
> !Name
> !Comment
> |-
> |Uint32||4||flags||standard flags (1-to-1 with standard flags in Bitcoin Core)
> |-
> |VarInt||1-8||msglen||Number of bytes in message string, excluding NUL
> termination
> |-
> |Char*||[msglen]||msg||The message being signed for all subjects,
> excluding NUL termination
> |-
> |Uint8||1||entries||Number of proof entriesWhy support
> multiple proofs? In particular with proof of funds, it is
> non-trivial to check a large number of individual proofs (one per
> UTXO) for duplicates. Software could be written to do so, but it seems
> more efficient to build this check into the specification
> itself.
> |}
>
> The above is followed by [entries] number of signature entries:
>
> {|class="wikitable" style="text-align: center;"
> |-
> !Type
> !Length
> !Name
> !Comment
> |-
> |VarInt||1-8||scriptsiglen||Number of bytes in scriptSig data
> |-
> |Uint8*||[scriptsiglen]||scriptsig||ScriptSig data
> |-
> |VarInt||1-8||witlen||Number of bytes in witness program data
> |-
> 

Re: [bitcoin-dev] Suggestion for a universal bitcoin value scale

[Karl-Johan Alm via bitcoin-dev]

A potential problem is that it would be a new attack vector to simply
color something to appear as e.g. 10x more than it really is, if
everyone started using this system.
On Sun, Aug 19, 2018 at 5:27 AM Martin Damgaard via bitcoin-dev
 wrote:
>
> Hi bitcoin-dev@lists.linuxfoundation.org
>
> Here is my humble attempt to make a contribution to the impressive work that 
> you all are doing.
>
> I am unfamiliar with the normal BIP procedures. I have therefore just tried 
> to follow the example of BIP 176 by Jimmy Song, in order make something 
> similar. I suggest a universal bitcoin value color scale, for tackling the 
> same decimal problem, as identified by the BIP 176 proposal.
>
> I have attached the document in three different formats (*.rtf, *.pdf and 
> *.docx) as I do not know your preferred format. I hope you will find my 
> suggestion useful.
>
>
>
> Thank you and all the best
>
> Martin Damgaard
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] RFC: BIP 322: Generic Signed Message Format

[Karl-Johan Alm via bitcoin-dev]

Hi.

[note: BIP number was assigned to PR before this email was sent; I did
not self-assign the BIP number]

Below is a proposal to extend the existing sign/verifymessage format
to a more generalized variant relying on the script verification
mechanism in Bitcoin itself for message signing/verification, based on
the original discussion
(https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-March/015818.html)
.

PR is here: https://github.com/bitcoin/bips/pull/725

A formatted version of this text can be seen here:
https://github.com/kallewoof/bips/blob/bip-generic-signmessage/bip-generic-signmessage.mediawiki

Note: I am not sure how to best deal with CLTV/CSV stuff here, ultimately.

Note 2: I have received suggestions from several people to use a
Bitcoin transaction instead. If someone could explain why this is
beneficial, it would be very helpful. I'm not against it, just feels
like the whole transaction part is unnecessary complexity/overhead.

---

  BIP: 322
  Layer: Applications
  Title: Generic Signed Message Format
  Author: Karl-Johan Alm 
  Comments-Summary: No comments yet.
  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322
  Status: Draft
  Type: Standards Track
  Created: 2018-09-10
  License: CC0-1.0


== Abstract ==

A standard for interoperable generic signed messages based on the
Bitcoin Script format.

== Motivation ==

The current message signing standard only works for P2PKH (1...)
addresses. By extending it to use a Bitcoin Script based approach, it
could be made more generic without causing a too big burden on
implementers, who most likely have access to Bitcoin Script
interpreters already.

== Specification ==

A new structure SignatureProof is added, which is a
simple serializable scriptSig & witnessProgram container.

Two actions "Sign" and "Verify" are defined.

=== SignatureProof container ===

{|class="wikitable" style="text-align: center;"
|-
!Type
!Length
!Name
!Comment
|-
|Uint32||4||flags||standard flags (1-to-1 with standard flags in Bitcoin Core)
|-
|VarInt||1-8||msglen||Number of bytes in message string, excluding NUL
termination
|-
|Char*||[msglen]||msg||The message being signed for all subjects,
excluding NUL termination
|-
|Uint8||1||entries||Number of proof entriesWhy support
multiple proofs? In particular with proof of funds, it is
non-trivial to check a large number of individual proofs (one per
UTXO) for duplicates. Software could be written to do so, but it seems
more efficient to build this check into the specification
itself.
|}

The above is followed by [entries] number of signature entries:

{|class="wikitable" style="text-align: center;"
|-
!Type
!Length
!Name
!Comment
|-
|VarInt||1-8||scriptsiglen||Number of bytes in scriptSig data
|-
|Uint8*||[scriptsiglen]||scriptsig||ScriptSig data
|-
|VarInt||1-8||witlen||Number of bytes in witness program data
|-
|Uint8*||[witlen]||wit||Witness program
|}

In some cases, the scriptsig may be empty (scriptsiglen=0).

=== Signing ===

The "Sign" action takes as input a scriptPubKey and a message (e.g.
"hello world"). It succeeds or fails.

# FAIL if scriptPubKey already exists in scriptPubKeys set, otherwise
insert itWhy track duplicates? Because a 3-entry
proof is not proving 3 scriptPubKeys unless they are all distinct, or
unless they are proving different UTXO:s (see Future Extensions)
# Derive the private key privkey for the scriptPubKey, or FAIL
# Define the message pre-image as the sequence "Bitcoin Message:"
concatenated with the message, encoded in UTF-8 using Normalization
Form Compatibility Decomposition (NFKD)
# Let sighash = sha256(sha256(scriptPubKey || pre-image))
# Generate a signature sig with privkey=privkey, sighash=sighash

Repeat the above operation for each scriptPubKey, retaining the
scriptPubKeys set. As noted, if the same scriptPubKey appears more
than once, the sign operation must fail.

=== Verifying ===

The "Verify" action takes as input a standard flags value, a
scriptPubKey, a message, a script sig, and a witness program.
It emits one of INCONCLUSIVE, VALID, INVALID, or ERROR.

# Return ERROR if scriptPubKey already exists in scriptPubKeys set,
otherwise insert it
# If one or more of the standard flags are unknown, return INCONCLUSIVE
# Define the message pre-image as the sequence "Bitcoin Message:"
concatenated with the message, encoded in UTF-8 using Normalization
Form Compatibility Decomposition (NFKD).
# Let sighash = sha256(sha256(scriptPubKey || pre-image))
# Verify Script with flags=standard flags, scriptSig=script sig,
scriptPubKey=scriptPubKey, witness=witness program, and
sighash=sighash
# Return VALID if verify succeeds, otherwise return INVALID

Repeat the above operation for each scriptPubKey, retaining the
scriptPubKeys set. As noted, if the same scriptPubKey appears more
than once, the verify operation must fail with an ERROR.

* If a verification call returns ERROR or INVALID, return ERROR or
INVALID immediately, ignoring as yet unverified entries.

Re: [bitcoin-dev] Testnet3 Reest

[Karl-Johan Alm via bitcoin-dev]

On Fri, Aug 31, 2018 at 9:43 PM Gregory Maxwell via bitcoin-dev
 wrote:
> We looked at doing this previously in Bitcoin core and jtimon had some
> patches,  but the existing approach increased the size of the
> blockindex objects in memory  while not in signed testnet mode.   This
> could probably have been fixed by turning one of the fields like the
> merkel root into a union of it's normal value and a pointer a
> look-aside block index that is used only in signed block testnet mode.

I am currently working on an implementation that simply puts a global
mapping of block hash to signature that is transparently
(de)serialized in the block header.

We were looking into various ways to stuff the signature into the
actual header itself without changing its size, but this looked like
it required truncating the prevblock/merkleroots and such, which
seemed a bit too invasive.

I don't think my approach with a global mapping to sig differs in any
meaningful way from your suggested union, but corrections welcome.

The code is here: https://github.com/kallewoof/bitcoin/tree/signet

I believe jtimon is interested in helping out, and Jeremy Rubin has
also said he wants to help.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size

[Karl-Johan Alm via bitcoin-dev]

On Tue, Jun 5, 2018 at 10:08 AM, Jim Posen  wrote:
> It also derives all bandwidth gains from address reuse. So I'm
> hesitant to make the complexity tradeoff for bandwidth savings due to a
> behavior that is actively discouraged.

I don't understand this comment. The bandwidth gains are not from
address reuse, they are from the observed property that false
positives are independent between two filters. I.e. clients that
connect once a day will probably download 2-3 filters at most, if they
had nothing relevant in the last ~144 blocks.

-Kalle.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size

[Karl-Johan Alm via bitcoin-dev]

On Fri, May 18, 2018 at 12:25 AM, Matt Corallo via bitcoin-dev
 wrote:
> In general, I'm concerned about the size of the filters making existing
> SPV clients less willing to adopt BIP 158 instead of the existing bloom
> filter garbage and would like to see a further exploration of ways to
> split out filters to make them less bandwidth intensive. Some further
> ideas we should probably play with before finalizing moving forward is
> providing filters for certain script templates, eg being able to only
> get outputs that are segwit version X or other similar ideas.

There is also the idea of multi-block filters. The idea is that light
clients would download a pair of filters for blocks X..X+255 and
X+256..X+511, check if they have any matches and then grab pairs for
any that matched, e.g. X..X+127 & X+128..X+255 if left matched, and
iterate down until it ran out of hits-in-a-row or it got down to
single-block level.

This has an added benefit where you can accept a slightly higher false
positive rate for bigger ranges, because the probability of a specific
entry having a false positive in each filter is (empirically speaking)
independent. I.e. with a FP probability of 1% in the 256 range block
and a FP probability of 0.1% in the 128 range block would mean the
probability is actually 0.001%.

Wrote about this here: https://bc-2.jp/bfd-profile.pdf (but the filter
type is different in my experiments)
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Few questions regarding ListTransaction

[Karl-Johan Alm via bitcoin-dev]

Thanks for clarifying!

On Wed, Apr 11, 2018 at 6:48 PM, ZmnSCPxj  wrote:
> Good morning Karl-Johan Alm,
>
> To clarify:
>
> Nothing prevents a miner from completely ignoring nSequence when putting 
> transactions in blocks.
>
> Unconfirmed transactions are, by definition, not recorded in blocks.  So if 
> there is a transaction 0xFFF nSequence and fee 1000 satoshi, and another 
> conflicting transaction 0xFFF nSequence and fee 1 satoshi, miners 
> can include the latter one even if the first one came to their knowledge 
> first, regardless nSequence.
>
> Thus, in the end "full replace-by-fee", where nSequence is IGNORED for 
> purposes of replace-by-fee, is expected to become the norm, and we should 
> really be designing our wallets and so on so that we only trust transactions 
> that have confirmations.
>
> The "nSequence=0x means opt-OUT of RBF" convention is only followed 
> by fullnodes running standard bitcoind.  Nothing prevents miners from running 
> patched bitcoind that ignores this rule, and connecting with similar peers 
> who also ignore this rule.
>
> Regards,
> ZmnSCPxj
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐ Original Message ‐‐‐
>
> On April 11, 2018 5:37 PM, Peter Todd via bitcoin-dev 
>  wrote:
>
>> On Wed, Apr 11, 2018 at 05:10:43PM +0900, Karl-Johan Alm wrote:
>>
>> > On Wed, Apr 11, 2018 at 4:52 PM, Peter Todd p...@petertodd.org wrote:
>> >
>> > > Or via full replace-by-fee, which appears to be used by a significant 
>> > > minority
>> > >
>> > > of miners:
>> >
>> > I was of the impression that final transactions (sequence=0x)
>> >
>> > cannot be RBF'd.
>>
>> My full-replace-by-fee tree ignores that. It also does preferential peering 
>> to
>>
>> ensure it's well connected with likewise peers, and thus the whole network.
>>
>>
>> ---
>>
>> https://petertodd.org 'peter'[:-1]@petertodd.org
>>
>> bitcoin-dev mailing list
>>
>> bitcoin-dev@lists.linuxfoundation.org
>>
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Few questions regarding ListTransaction

[Karl-Johan Alm via bitcoin-dev]

On Wed, Apr 11, 2018 at 4:52 PM, Peter Todd  wrote:
> Or via full replace-by-fee, which appears to be used by a significant minority
> of miners:

I was of the impression that final transactions (sequence=0x)
cannot be RBF'd.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Few questions regarding ListTransaction

[Karl-Johan Alm via bitcoin-dev]

Clarification on one part below:

On Wed, Apr 11, 2018 at 2:21 PM, Karl-Johan Alm
 wrote:
> On Wed, Apr 11, 2018 at 5:29 AM, Maksim Solovjov via bitcoin-dev
>  wrote:
>> 1. What does it mean for a transaction ( with 0 confirmations ) to be
>> trusted or not?
>
> It is trusted if (1) it is final (i.e. it can't be replaced), (2) it
> is not in a block that was reorged out (negative confirmation count),
> (3) the 'spend zero conf change' option is set, (4) it is in the
> mempool, and (5) all inputs are from us.

"can't be replaced" here means it cannot be replaced through
conventional means. It is always possible to replace a transaction
that has not yet been confirmed, e.g. by asking a miner to mine a
conflicting transaction directly.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Few questions regarding ListTransaction

[Karl-Johan Alm via bitcoin-dev]

On Wed, Apr 11, 2018 at 5:29 AM, Maksim Solovjov via bitcoin-dev
 wrote:
> 1. What does it mean for a transaction ( with 0 confirmations ) to be
> trusted or not?

It is trusted if (1) it is final (i.e. it can't be replaced), (2) it
is not in a block that was reorged out (negative confirmation count),
(3) the 'spend zero conf change' option is set, (4) it is in the
mempool, and (5) all inputs are from us.

> 2. When does confirmations can be -1 ( conflicted )?
> What does it mean to have conflicted transaction?
> Is it about Transaction Malleability? Double Spend? or both?

A transaction is conflicted if a different transaction exists that
spends the same inputs. A transaction gets -N confirmations if it is
mined in a block, and that block is orphaned away, and a different
transaction is mined in the new block so that the transaction becomes
a double spend.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] proposal: extend WIF format for segwit

[Karl-Johan Alm via bitcoin-dev]

Hello,

I made slight modification to the BIP, dropping the 0x80 jump to 0x10:
https://github.com/kallewoof/bips/blob/bip-typed-wif/bip-extended-privkey.mediawiki

I will make the corresponding changes to the reference implementation shortly.

If there are no objections I would also like to request a BIP number.


On Wed, Apr 4, 2018 at 3:06 PM, Karl Johan Alm
 wrote:
> I took the liberty of turning this into a BIP proposal -- the
> formatted version can be seen here:
> https://github.com/kallewoof/bips/blob/bip-typed-wif/bip-extended-privkey.mediawiki
>
> 
>   BIP: XXX
>   Layer: Applications
>   Title: Typed Private Keys
>   Author: Karl-Johan Alm 
>   Comments-Summary: No comments yet.
>   Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-XXX
>   Status: Draft
>   Type: Standards Track
>   Created: 2018-04-04
>   License: CC0-1.0
> 
>
> == Abstract ==
>
> An extension to the private key (WIF) format to specify what kind of
> public key the private key corresponds to.
>
> == Motivation ==
>
> There are several types of public keys which can all be associated
> with a given private key: P2PKH (legacy 1... format),
> P2SH-P2WPKH (SegWit public key inside P2SH), P2WPKH (bech32), etc.
>
> While private keys have a 1-byte suffix indicating whether the
> corresponding public key is compressed (0x01) or not
> (0x00), there is no way of knowing what kind of public
> keys were associated with the private key. As a result, when importing
> a private key, the wallet has to assume all kinds, and keep track of
> each possible alternative.
>
> By extending the suffix, we can specify what kind of public key was
> associated with the given private key.
>
> == Specification ==
>
> Currently, private keys are stored as a uint256 (private key data)
> followed by a uint8 (compressed flag). The latter is extended to
> specify the public key types:
>
> {|class="wikitable" style="text-align: center;"
> |-
> !Value
> !Type
> !Compr
> !Clarification
> |-
> |0x00||P2PKH_UNCOMPRESSED||No||Uncompressed legacy public
> key. Unknown public key format
> |-
> |0x01||P2PKH_COMPRESSED||Yes||Compressed legacy public
> key. Unknown public key format
> |-
> |0x80||P2PKH||Yes||Compressed legacy public key. Legacy
> public key format (1...)
> |-
> |0x81||P2WPKH||Yes||Bech32 format (native Segwit)
> |-
> |0x82||P2WPKH_P2SH||Yes||Segwit nested in BIP16 P2SH
> (3...)
> |-
> |0x85||P2SH||  ||Non-Segwit BIP16 P2SH (3...)
> |-
> |0x86||P2WSH||  ||Native Segwit P2SH
> |-
> |0x87||P2WSH_P2SH||  ||Native Segwit P2SH nested
> in BIP16 P2SH
> |}
>
> When a wallet imports a private key, it will have two outcomes:
>
> * the key is using one of the legacy types, in which case all types
> must be accounted for
> * the key is using one of the extended types, in which case the wallet
> need only track the specific corresponding public key
>
> == Rationale ==
>
> TODO
>
> == Compatibility ==
>
> This proposal is not backwards compatible, in that software that does
> not recognize the new types will not understand the compressed flag.
> It would be trivial to change this, by keeping the 'uncompressed'
> state as it is (0) and changing 'compressed' to be 'anything not 0',
> as opposed to 'the value 1'.
>
> The proposal *is* backwards compatible in that new wallet software
> will always understand the old WIF format, however. It will, as it
> does today, assume that any kind of public key is possible, and will
> have to track all of them, as it has to today.
>
> == Acknowledgements ==
>
> This BIP is based on the initial proposal by Thomas Voegtlin
>  on the Bitcoin Dev mailing
> listhttps://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015007.html
> and the Electrum 3.0
> implementationhttps://github.com/spesmilo/electrum/blob/82e88cb89df35288b80dfdbe071da74247351251/RELEASE-NOTES#L95-L108
>
> == Reference implementation ==
>
> There is a partial implementation which adds, but does not use, the
> types described in this BIP here:
> https://github.com/bitcoin/bitcoin/pull/12869
>
> == References ==
>
> 
>
> == Copyright ==
>
> This document is licensed under the Creative Commons CC0 1.0 Universal 
> license.
>
> On Mon, Sep 18, 2017 at 12:36 AM, Mark Friedenbach via bitcoin-dev
>  wrote:
>> Bech32 and WIF payload format are mostly orthogonal issues. You can design a
>> new wallet import format now and later switch it to Bech32.
>>
>> On Sep 17, 2017, at 7:42 AM, AJ West via bitcoin-dev
>>  wrote:
>>
>> Hi I have a small interjection about the point on error correction (excuse
>> me if it seems elementary). Isn't there an argument to be made where a
>> wallet software should never attempt to figure out the 'correct' address, or
>> in this case private key? I don't think it's crazy to suggest somebody could
>> import a slightly erroneous WIF, the software gracefully error-corrects 

Re: [bitcoin-dev] proposal: extend WIF format for segwit

[Karl Johan Alm via bitcoin-dev]

I took the liberty of turning this into a BIP proposal -- the
formatted version can be seen here:
https://github.com/kallewoof/bips/blob/bip-typed-wif/bip-extended-privkey.mediawiki


  BIP: XXX
  Layer: Applications
  Title: Typed Private Keys
  Author: Karl-Johan Alm 
  Comments-Summary: No comments yet.
  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-XXX
  Status: Draft
  Type: Standards Track
  Created: 2018-04-04
  License: CC0-1.0


== Abstract ==

An extension to the private key (WIF) format to specify what kind of
public key the private key corresponds to.

== Motivation ==

There are several types of public keys which can all be associated
with a given private key: P2PKH (legacy 1... format),
P2SH-P2WPKH (SegWit public key inside P2SH), P2WPKH (bech32), etc.

While private keys have a 1-byte suffix indicating whether the
corresponding public key is compressed (0x01) or not
(0x00), there is no way of knowing what kind of public
keys were associated with the private key. As a result, when importing
a private key, the wallet has to assume all kinds, and keep track of
each possible alternative.

By extending the suffix, we can specify what kind of public key was
associated with the given private key.

== Specification ==

Currently, private keys are stored as a uint256 (private key data)
followed by a uint8 (compressed flag). The latter is extended to
specify the public key types:

{|class="wikitable" style="text-align: center;"
|-
!Value
!Type
!Compr
!Clarification
|-
|0x00||P2PKH_UNCOMPRESSED||No||Uncompressed legacy public
key. Unknown public key format
|-
|0x01||P2PKH_COMPRESSED||Yes||Compressed legacy public
key. Unknown public key format
|-
|0x80||P2PKH||Yes||Compressed legacy public key. Legacy
public key format (1...)
|-
|0x81||P2WPKH||Yes||Bech32 format (native Segwit)
|-
|0x82||P2WPKH_P2SH||Yes||Segwit nested in BIP16 P2SH
(3...)
|-
|0x85||P2SH||  ||Non-Segwit BIP16 P2SH (3...)
|-
|0x86||P2WSH||  ||Native Segwit P2SH
|-
|0x87||P2WSH_P2SH||  ||Native Segwit P2SH nested
in BIP16 P2SH
|}

When a wallet imports a private key, it will have two outcomes:

* the key is using one of the legacy types, in which case all types
must be accounted for
* the key is using one of the extended types, in which case the wallet
need only track the specific corresponding public key

== Rationale ==

TODO

== Compatibility ==

This proposal is not backwards compatible, in that software that does
not recognize the new types will not understand the compressed flag.
It would be trivial to change this, by keeping the 'uncompressed'
state as it is (0) and changing 'compressed' to be 'anything not 0',
as opposed to 'the value 1'.

The proposal *is* backwards compatible in that new wallet software
will always understand the old WIF format, however. It will, as it
does today, assume that any kind of public key is possible, and will
have to track all of them, as it has to today.

== Acknowledgements ==

This BIP is based on the initial proposal by Thomas Voegtlin
 on the Bitcoin Dev mailing
listhttps://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015007.html
and the Electrum 3.0
implementationhttps://github.com/spesmilo/electrum/blob/82e88cb89df35288b80dfdbe071da74247351251/RELEASE-NOTES#L95-L108

== Reference implementation ==

There is a partial implementation which adds, but does not use, the
types described in this BIP here:
https://github.com/bitcoin/bitcoin/pull/12869

== References ==



== Copyright ==

This document is licensed under the Creative Commons CC0 1.0 Universal license.

On Mon, Sep 18, 2017 at 12:36 AM, Mark Friedenbach via bitcoin-dev
 wrote:
> Bech32 and WIF payload format are mostly orthogonal issues. You can design a
> new wallet import format now and later switch it to Bech32.
>
> On Sep 17, 2017, at 7:42 AM, AJ West via bitcoin-dev
>  wrote:
>
> Hi I have a small interjection about the point on error correction (excuse
> me if it seems elementary). Isn't there an argument to be made where a
> wallet software should never attempt to figure out the 'correct' address, or
> in this case private key? I don't think it's crazy to suggest somebody could
> import a slightly erroneous WIF, the software gracefully error-corrects any
> problem, but then the user copies that error onward such as in their backup
> processes like a paper wallet. I always hate to advocate against a feature,
> I'm just worried too much error correcting removes the burden of exactitude
> and attention of the user (eg. "I know I can have up to 4 errors").
>
> I'm pretty sure I read those arguments somewhere in a documentation or issue
> tracker/forum post. Maybe I'm misunderstanding the bigger picture in this
> particular case, but I was just reminded of that concept (even if it only
> applies generally).
>
> Thanks,
> AJ West
>
> On Sun, Sep 17, 2017 at 4:10 AM, Thomas 

Re: [bitcoin-dev] {sign|verify}message replacement

[Karl Johan Alm via bitcoin-dev]

Pieter,

Thanks for the feedback. Comments below:

On Mon, Mar 26, 2018 at 5:53 PM, Pieter Wuille  wrote:
> One solution is to include a version number in the signature, which
> explicitly corresponds to a set of validation flags. When the version number
> is something a verifier doesn't know, it can be reported as inconclusive
> (it's relying on features you don't know about).
>
> An solution is to verify twice; once with all consensus rules you know
> about, and once with standardness rules. If they're both valid, the
> signature is valid. If they're both invalid, the signature is invalid. If
> they're different (consensus valid but standardness invalid), you report the
> signature validation as inconclusive (it's relying on features you don't
> know about). This approach works as long as new features only use previous
> standardness-invalid scripts, but perhaps a version number is still needed
> to indicate the standardness flags.

I think the double verify approach seems promising. I assume old nodes
consider new consensus rule enforcing transactions as non-standard but
valid. If this is always the case, it may be an idea to simply fail
verification with a message indicating the node is unable to verify
due to unknown consensus rules.

>> RPC commands:
>>
>> sign   [=false]
>
> Why not extend the existing signmessage/verifymessage RPC? For legacy
> addresses it can fall back to the existing signature algorithm, while using
> the script-based approach for all others.

Yes, I initially thought it would be better to not use it as the
legacy behavior could be depended on god knows where, but I think
adding a legacy mode or simply doing the old way for 1xx is
sufficient.

>> (**) If  is true,  is the sighash, otherwise
>> sighash=sha256d(message).
>
>
> That's very dangerous I'm afraid. It could be used to trick someone into
> signing off on an actual transaction, if you get them to sign a "random
> looking" prehashed message. Even if you have a prehashed message, there is
> no problem with treating it as hex input to a second hashing step, so I
> think the prehashed option isn't needed. It's why the existing message
> signing functionality always forcibly prefixes "Bitcoin signed message:", to
> avoid signing something that unintentionally corresponds to a message
> intended for another goal.

Eek.. good point...
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] {sign|verify}message replacement

[Karl Johan Alm via bitcoin-dev]

On Fri, Mar 16, 2018 at 1:59 AM, Greg Sanders  wrote:
> Sorry if I missed the rationale earlier, but why not just do a transaction,
> with a FORKID specifically for this? Then a node can have a mempool
> acceptance test that returns true even if the signature is not valid as per
> Bitcoin consensus, but only due to the FORKID?
>
> This way basically any wallet can support this provided generic FORKID
> support.

You'd basically have to provide an entire transaction rather than just
the signature, so there's some overhead. (Copy-pasting may become
unwieldy quicker.)
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] {sign|verify}message replacement

[Karl Johan Alm via bitcoin-dev]

On Thu, Mar 15, 2018 at 2:14 PM, Luke Dashjr  wrote:
> Not necessarily specific UTXOs (that would contradict fungibility, as well as
> be impossible for hot/cold wallet separation), but just to prove funds are
> available. The current sign message cannot be used to prove present possession
> of funds, only that you receive funds.

By saying "not necessarily specific UTXOs", are you saying it may be
spent outputs? I'm a little confused I think.

On Thu, Mar 15, 2018 at 8:53 PM, Jim Posen  wrote:
> In this general signing-a-script context, I think a verifier might want to
> see the time conditions under which it may be spent. The proof container
> could include an optional nLockTime which defaults to 0 and nSequence which
> defaults to 0x...

Good point!

>> I think it would just use the default (SIGHASH_ALL?) for simplicity.
>> Is there a good reason to tweak it?
>
> I took another look and there should definitely be a byte appended to the
> end of the sig so that the encoding checks pass, but I think it might as
> well be a 0x00 byte since it's not actually a sighash flag.

I think the sighash flag affects the outcome of the actual
verification, but I could be mistaken.

-Kalle.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] {sign|verify}message replacement

[Karl Johan Alm via bitcoin-dev]

On Wed, Mar 14, 2018 at 12:36 PM, Luke Dashjr  wrote:
> Ideally, it should support not only just "proof I receive at this address",
> but also "proof of funds" (as a separate feature) since this is a popular
> misuse of the current message signing (which doesn't actually prove funds at
> all). To do this, it needs to be capable of signing for multiple inputs.

Re-reading this, I think what you mean is it should be possible to
create a proof for (a) specific UTXO(s), hence "inputs". That sounds
pretty useful, yeah!

So you could provide a mix of addresses and inputs (as txid:vout) and
it would generate a proof that signs the message for each input
(taking scriptPubKey from address or from the UTXO data directly on
the blockchain).

-Kalle.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] {sign|verify}message replacement

[Karl Johan Alm via bitcoin-dev]

On Thu, Mar 15, 2018 at 6:43 AM, Jim Posen  wrote:
> How are scripts with OP_CLTV and OP_CSV handled by verifiers? Do they always
> succeed? Or should an nLockTime and nSequence also be included in the proof
> in a way that can be parsed out and displayed to verifiers?

Good question.. Since you don't really have the input(s), I think it's
fine to always assume sufficient time/height on CLTV/CSV checks.

> I assume any signatures in the scriptSig/witness data would have no sighash
> type?

I think it would just use the default (SIGHASH_ALL?) for simplicity.
Is there a good reason to tweak it?

-Kalle.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] {sign|verify}message replacement

[Karl Johan Alm via bitcoin-dev]

On Wed, Mar 14, 2018 at 5:46 AM, Kalle Rosenbaum  wrote:
> I can't really see from your proposal if you had thought of this: A soft
> fork can make old nodes accept invalid message signatures as valid. For
> example, a "signer" can use a witness version unknown to the verifier to
> fool the verifier. Witness version is detectable (just reject unknown
> witness versions)  but there may be more subtle changes. Segwit was not
> "detectable" in that way, for example.
>
> This is the reason why I withdrew BIP120. If you have thought about the
> above, I'd be very interested.

I'm not sure I see the problem. The scriptPubKey is derived directly
from the address in all cases, which means the unknown witness version
would have to be committed to in the address itself.

So yeah, I can make a P2SH address with a witness version > 0 and a to
me unknown pubkey and then fool you into thinking I own it, but I
don't really see why you'd ultimately care. In other words, if I can
SPEND funds sent to that address today, I can prove that I can spend
today, which is the purpose of the tool, I think.

For the case where the witness version HAS been upgraded, the above
still applies, but I'm not sure it's a big issue. And it doesn't
really require an old node. I just need to set witness version >
current witness version and the problem applies to all nodes.

On Wed, Mar 14, 2018 at 8:36 AM, Luke Dashjr  wrote:
> I don't see a need for a new RPC interface, just a new signature format.

All right.

> Ideally, it should support not only just "proof I receive at this address",
> but also "proof of funds" (as a separate feature) since this is a popular
> misuse of the current message signing (which doesn't actually prove funds at
> all). To do this, it needs to be capable of signing for multiple inputs.

I assume by inputs you mean addresses/keys. The address field could
optionally be an array. That'd be enough?

> Preferably, it should also avoid disclosing the public key for existing or
> future UTXOs. But I don't think it's possible to avoid this without something
> MAST-like first. Perhaps it can be a MAST upgrade later on, but the new
> signature scheme should probably be designed with it in mind.

I'd love to not have to reveal the public key, but I'm not sure how it
would be done, even with MAST.

On Wed, Mar 14, 2018 at 12:12 PM, Anthony Towns  wrote:
> Wouldn't it be sufficient for old nodes to check for standardness of the 
> spending script and report non-standard scripts as either invalid outright, 
> or at least highly questionable? That should prevent confusion as long as 
> soft forks are only making nonstandard behaviours invalid.

That seems sensible to me. A warning would probably be useful, in case
the verifier is running old software.

-Kalle.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] {sign|verify}message replacement

[Karl Johan Alm via bitcoin-dev]

Hello,

I am considering writing a replacement for the message signing tools
that are currently broken for all but the legacy 1xx addresses. The
approach (suggested by Pieter Wuille) is to do a script based
approach. This does not seem to require a lot of effort for
implementing in Bitcoin Core*. Below is my proposal for this system:

A new structure SignatureProof is added, which is a simple scriptSig &
witnessProgram container that can be serialized. This is passed out
from/into the signer/verifier.

RPC commands:

sign   [=false]

Generates a signature proof for  using the same method that
would be used to spend coins sent to .**

verify[=false]

Deserializes and executes the proof using a custom signature checker
whose sighash is derived from . Returns true if the check
succeeds, and false otherwise. The scriptPubKey is derived directly
from .**

Feedback welcome.

-Kalle.

(*) Looks like you can simply use VerifyScript with a new signature
checker class. (h/t Nicolas Dorier)
(**) If  is true,  is the sighash, otherwise
sighash=sha256d(message).
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Mempool optimized fees, etc. (Scaling Bitcoin)

[Karl Johan Alm via bitcoin-dev]

This is the paper detailing the research behind my talk "Optimizing
fee estimation via the mempool state" (the presentation only covers
part of the paper) at Scaling Stanford (this coming Sunday). Feedback
welcome.

https://bc-2.jp/mempool.pdf
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Merkle branch verification & tail-call semantics for generalized MAST

[Karl Johan Alm via bitcoin-dev]

On Wed, Sep 13, 2017 at 4:57 AM, Mark Friedenbach via bitcoin-dev
 wrote:
>> Without the limit I think we would be DoS-ed to dead
>
> 4MB of secp256k1 signatures takes 10s to validate on my 5 year old
> laptop (125,000 signatures, ignoring public keys and other things that
> would consume space). That's much less than bad blocks that can be
> constructed using other vulnerabilities.

Sidenote-ish, but I also believe it would be fairly trivial to keep a
per UTXO tally and demand additional fees when trying to respend a
UTXO which was previously "spent" with an invalid op count. I.e. if
you sign off on an input for a tx that you know is bad, the UTXO in
question will be penalized proportionately to the wasted ops when
included in another transaction later. That would probably kill that
DoS attack as the attacker would effectively lose bitcoin every time,
even if it was postponed until they spent the UTXO. The only thing
clients would need to do is to add a fee rate penalty ivar and a
mapping of outpoint to penalty value, probably stored as a separate
.dat file. I think.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Updating the Scaling Roadmap

[Karl Johan Alm via bitcoin-dev]

On Wed, Jul 12, 2017 at 6:11 AM, Gregory Maxwell via bitcoin-dev
 wrote:
> IMO the way to do "roadmaps" in Bitcoin is to roadmap the finalization
> and release process once the basic technology is done; because it's
> only past that point that guarantees can really start being made.

Bitcoin development differs from Linux kernel development in a number
of obvious ways, such as the fact Bitcoin is being "patched in
flight". The current political situation over Bitcoin development is
also quite different, with scalability being a major concern for a lot
of users, and conflicting views leading to risky technical gambles.

Having *something* like a roadmap that gives the average user some
insights into what exactly is being planned for Bitcoin is very
desirable, arguably even necessary, in particular for the scaling
solutions. Putting deadlines and dates in would of course be highly
irresponsible, as no one can predict how much of their free time
volunteer developers will put into the project in advance (or whether
they will stick around for the next X months or stop being
contributors).

I think there is necessity for a document that describes the project
intentions for scaling solutions, but I don't think adding dates and
deadlines is appropriate. That may or may not be a roadmap. I imagine
such a document would be updated regularly as appropriate, which means
it may be less of a roadmap than the traditional kind.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] User Activated Soft Fork Split Protection

[Karl Johan Alm via bitcoin-dev]

One thing about BIP148 activation that may be affected by this is the
fact that segwit signalling non-BIP148 miners + BIP148 miners may hold
majority hash power and prevent a chain split. With this SF, that will
no longer be the case, right? Or am I completely confused on the
subject?

On Wed, Jun 7, 2017 at 9:56 AM, James Hilliard via bitcoin-dev
 wrote:
> Due to the proposed calendar(https://segwit2x.github.io/) for the
> SegWit2x agreement being too slow to activate SegWit mandatory
> signalling ahead of BIP148 using BIP91 I would like to propose another
> option that miners can use to prevent a chain split ahead of the Aug
> 1st BIP148 activation date.
>
> The splitprotection soft fork is essentially BIP91 but using BIP8
> instead of BIP9 with a lower activation threshold and immediate
> mandatory signalling lock-in. This allows for a majority of miners to
> activate mandatory SegWit signalling and prevent a potential chain
> split ahead of BIP148 activation.
>
> This BIP allows for miners to respond to market forces quickly ahead
> of BIP148 activation by signalling for splitprotection. Any miners
> already running BIP148 should be encouraged to use splitprotection.
>
> 
>   BIP: splitprotection
>   Layer: Consensus (soft fork)
>   Title: User Activated Soft Fork Split Protection
>   Author: James Hilliard 
>   Comments-Summary: No comments yet.
>   Comments-URI:
>   Status: Draft
>   Type: Standards Track
>   Created: 2017-05-22
>   License: BSD-3-Clause
>CC0-1.0
> 
>
> ==Abstract==
>
> This document specifies a coordination mechanism for a simple majority
> of miners to prevent a chain split ahead of BIP148 activation.
>
> ==Definitions==
>
> "existing segwit deployment" refer to the BIP9 "segwit" deployment
> using bit 1, between November 15th 2016 and November 15th 2017 to
> activate BIP141, BIP143 and BIP147.
>
> ==Motivation==
>
> The biggest risk of BIP148 is an extended chain split, this BIP
> provides a way for a simple majority of miners to eliminate that risk.
>
> This BIP provides a way for a simple majority of miners to coordinate
> activation of the existing segwit deployment with less than 95%
> hashpower before BIP148 activation. Due to time constraints unless
> immediately deployed BIP91 will likely not be able to enforce
> mandatory signalling of segwit before the Aug 1st activation of
> BIP148. This BIP provides a method for rapid miner activation of
> SegWit mandatory signalling ahead of the BIP148 activation date. Since
> the primary goal of this BIP is to reduce the chance of an extended
> chain split as much as possible we activate using a simple miner
> majority of 65% over a 504 block interval rather than a higher
> percentage. This BIP also allows miners to signal their intention to
> run BIP148 in order to prevent a chain split.
>
> ==Specification==
>
> While this BIP is active, all blocks must set the nVersion header top
> 3 bits to 001 together with bit field (1<<1) (according to the
> existing segwit deployment). Blocks that do not signal as required
> will be rejected.
>
> ==Deployment==
>
> This BIP will be deployed by "version bits" with a 65%(this can be
> adjusted if desired) activation threshold BIP9 with the name
> "splitprotecion" and using bit 2.
>
> This BIP starts immediately and is a BIP8 style soft fork since
> mandatory signalling will start on midnight August 1st 2017 (epoch
> time 1501545600) regardless of whether or not this BIP has reached its
> own signalling threshold. This BIP will cease to be active when segwit
> is locked-in.
>
> === Reference implementation ===
>
> 
> // Check if Segregated Witness is Locked In
> bool IsWitnessLockedIn(const CBlockIndex* pindexPrev, const
> Consensus::Params& params)
> {
> LOCK(cs_main);
> return (VersionBitsState(pindexPrev, params,
> Consensus::DEPLOYMENT_SEGWIT, versionbitscache) ==
> THRESHOLD_LOCKED_IN);
> }
>
> // SPLITPROTECTION mandatory segwit signalling.
> if ( VersionBitsState(pindex->pprev, chainparams.GetConsensus(),
> Consensus::DEPLOYMENT_SPLITPROTECTION, versionbitscache) ==
> THRESHOLD_LOCKED_IN &&
>  !IsWitnessLockedIn(pindex->pprev, chainparams.GetConsensus()) &&
> // Segwit is not locked in
>  !IsWitnessEnabled(pindex->pprev, chainparams.GetConsensus()) ) //
> and is not active.
> {
> bool fVersionBits = (pindex->nVersion & VERSIONBITS_TOP_MASK) ==
> VERSIONBITS_TOP_BITS;
> bool fSegbit = (pindex->nVersion &
> VersionBitsMask(chainparams.GetConsensus(),
> Consensus::DEPLOYMENT_SEGWIT)) != 0;
> if (!(fVersionBits && fSegbit)) {
> return state.DoS(0, error("ConnectBlock(): relayed block must
> signal for segwit, please upgrade"), REJECT_INVALID, "bad-no-segwit");
> }
> }
>
> // BIP148 mandatory segwit signalling.
> int64_t nMedianTimePast = pindex->GetMedianTimePast();
> if ( (nMedianTimePast >= 1501545600) &&  // Tue 01 Aug 2017 00:00:00 UTC
>  (nMedianTimePast <= 

Re: [bitcoin-dev] BIP Proposal: Compact Client Side Filtering for Light Clients

[Karl Johan Alm via bitcoin-dev]

On Sat, Jun 3, 2017 at 2:55 AM, Alex Akselrod via bitcoin-dev
 wrote:
> Without a soft fork, this is the only way for light clients to verify that
> peers aren't lying to them. Clients can request headers (just hashes of the
> filters and the previous headers, creating a chain) and look for conflicts
> between peers. If a conflict is found at a certain block, the client can
> download the block, generate a filter, calculate the header by hashing
> together the previous header and the generated filter, and banning any peers
> that don't match. A full node could prune old filters if you wanted and
> recalculate them as necessary if you just keep the filter header chain info
> as really old filters are unlikely to be requested by correctly written
> software but you can't guarantee every client will follow best practices
> either.

Ahh, so you actually make a separate digest chain with prev hashes and
everything. Once/if committed digests are soft forked in, it seems a
bit overkill but maybe it's worth it. (I was always assuming committed
digests in coinbase would come after people started using this, and
that people could just ask a couple of random peers for the digest
hash and ensure everyone gave the same answer as the hash of the
downloaded digest..).

> The simulations are based on completely random data within given parameters.

I noticed an increase in FP hits when using real data sampled from
real scriptPubKeys and such. Address reuse and other weird stuff. See
"lies.h" in github repo for experiments and chainsim.c initial part of
main where wallets get random stuff from the chain.

> I will definitely try to reproduce my experiments with Golomb-Coded
> sets and see what I come up with. It seems like you've got a little
> less than half the size of my digests for 1-block digests but I
> haven't tried making digests for all blocks (and lots of early blocks
> are empty).
>
>
> Filters for empty blocks only take a few bytes and sometimes zero when the
> coinbase output is a burn that doesn't push any data (example will be in the
> test vectors that I'll have ready shortly).

I created digests for all blocks up until block #469805 and actually
ended up with 5.8 GB, which is 1.1 GB lower than what you have, but
may be worse perf-wise on false positive rates and such.

> How fast are these to create? Would it make sense to provide digests
> on demand in some cases, rather than keeping them around indefinitely?
>
>
> They're pretty fast and can be pruned if desired, as mentioned above, as
> long as the header chain is kept.

For comparison, creating the digests above (469805 of them) took
roughly 30 mins on my end, but using the kstats format so probably
higher on an actual node (should get around to profiling that...).
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Block Filter Digest profiling

[Karl Johan Alm via bitcoin-dev]

Hello,

I have spent a fair bit of time trying to nail how exactly block
filter digests[1] should be done to optimize bandwidth, space,
resource usage.

The report can be found here: http://bc-2.jp/bfd-profile.pdf

This graph shows bandwidth use of 200 wallets simulated over 5000
blocks: http://bc-2.jp/bandwidth_bfd.png (black line is "sync once per
block" wallet, yellow is "sync once per 144 blocks" wallet, red is
average across all wallets).

An interesting insight made during the experiments: when allowing
digests to contain multiple blocks, the false positive rate of high
block count digests can be higher than normal, because the probability
of a false positive hit for a given entry in multiple digests,
assuming their sizes differ, is almost completely independent.

The results look rather promising to me, but I would like to hear
comments, in particular on the approach taken, if I made any faulty
assumptions, bad math mistakes, etc.

I am also curious what people consider to be acceptable costs in terms
of bandwidth use and memory (I couldn't find any stats on bandwidth
use of bloom filters). In the profiling, I restricted the field sizes
to 2^27 = 128 MB. I assumed this was appropriate as these fields are
very short lived, and in worst case, a client *could* do the scan and
decode simultaneously, without allocating up the space for the field
at all. For high block count digests (e.g. 1024 blocks), this is
sometimes overfilled. I wonder if 2^28 (256 MB) fields would be at all
acceptable or if an over-filled (high false positive rate) field is
better.

For that matter, I am not entirely sure 1024-block digests are
necessary, but they do come with an average 15 kb/block which is
pretty good.

I also wonder if the serialization approach taken is overkill or not.
It does save some space instead of simply storing "BBBA" but adds
some complexity that may not be warranted.

[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-May/012636.html
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] I do not support the BIP 148 UASF

[Karl Johan Alm via bitcoin-dev]

On Tue, May 23, 2017 at 1:03 PM, Steven Pine via bitcoin-dev
 wrote:
> Correct me if I am wrong, but currently core developers are arguing over
> whether or not to allow an optional configuration switch which defaults off
> but signals and enforces BIP148 when used. Who are we protecting users from,
> themselves? Are you protecting core? from what? I am somewhat genuinely
> befuddled by those who can't even allow a user config switch to be set.

Essentially, if we make a potentially very harmful option easy to
enable for users, we are putting them at risk, so yes, this is about
protecting users of the base Bitcoin Core implementation. Users have,
hopefully, come to appreciate this implementation for the peer
review-based strict development process, and making a hasty decision
due to time constraints (segwit activation expiration) may have
undesirable consequences. Opinions among the regular contributors are
split on the matter, which to me is an indication we should be
cautious and consider all aspects before making a decision on the
matter.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposal: Rate Limiting with server specified Proof of Work challenges

[Karl Johan Alm via bitcoin-dev]

Erik,

On Tue, May 9, 2017 at 3:58 AM, Erik Aronesty  wrote:
> - It would be cool if any rate-limiting POW was specified as bytecode ... so
> nodes can plug in as many "machine-captcha" things as they please, and
> solvers can choose to solve... or just say "nope too hard".

I'm not entirely sure what you mean, but right now you can make an
arbitrary chain of challenges, and the BIP includes methods for
determining an approximate time to solve (nodes will, at the very
least, discard any challenge which will on average take longer time to
solve than the expiration of the challenge itself, for example, i.e.
the "nope too hard" part).

> - Alternately, it would be a lot nicer if you just required people to pay a
> nanobit  that could prevent DDOS even better, and generate a revenue
> stream for nodes.

Others mentioned this approach. I haven't given it much thought.
Admittedly it would be an effective way to prevent DoS but it also has
some unwanted side effects that need to be cleared up (e.g. in a
no-gains scenario like the BIP proposes, the node requesting PoW done
doesn't *gain* anything from lying to the node performing the work).
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP Proposal: Rate Limiting with server specified Proof of Work challenges

[Karl Johan Alm via bitcoin-dev]

Hello,

I am proposing a new feature for rate limiting purposes where nodes
can make and solve arbitrary PoW challenges in return for connection
slots (to be expanded to cover e.g. bloom filters or other DoS risky
services).

The BIP currently includes two proofs of work (sha256 and
cuckoo-cycle) which can be combined (e.g. sha256(cuckoo-cycle) or
sha256(sha256(sha256)), etc).

Link: 
https://github.com/kallewoof/bips/blob/pow-connection-slots/bip-rate-limiting-via-pow.mediawiki

Feedback welcome.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev