Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Andrew Chow via bitcoin-dev]

Hi Dan,

I don't think nostr would be a suitable replacement for the mailing 
list, although this opinion is biased by the fact that I do not use 
nostr and find it to be uninteresting.

 From my limited understanding of how nostr works, it's not clear to me 
how a distributed system that uses message broadcast would work in the 
same way as a mailing list. How would people "subscribe"? How would 
archives be searched or otherwise be available to people who are not on 
nostr? How do you distinguish and filter between legitimate dev posts 
intended for discussion and random crap and shitposting as shows up on 
social media?

I also don't think that long form text on nostr (or any similar 
platform) can sufficiently replace email. None of these things seem to 
contain a way to have a separate subject line as email does. Subjects 
are immensely important for me as it provides a quick and easy way to 
filter out things I don't care about reading. I don't want to have read 
something in before I can decide that I don't care about reading it.

In general, I strongly prefer email, or a platform that has email as a 
first class user interface, over platforms such as nostr, matrix, or web 
forums. Email is universal - everyone has one and everyone knows how it 
works. It dramatically lowers the barrier of entry. Having to make an 
account somewhere or download some specific client in order to 
participate will simply result in only the most dedicated participating. 
Development in open source must be an open process and the barriers to 
entry should be low.

Lastly, IIRC the plan is to shut down the list by the end of this year. 
Any solution that requires custom software and bridges to be created are 
not going to be viable in this time frame.


Andrew

On 11/07/2023 12:03 PM, Ademan via bitcoin-dev wrote:
> Hi Bryan,
> 
> I don't really want my first (and last?) devlist message to be a fairly 
> off-the-cuff post on this topic, but here we go anyway.
> 
> At the risk of sounding like a nostr evangelist (I promise I'm not), I 
> want to suggest nostr as a potential replacement to the mailing list. A 
> decent chunk of software would need to be written, but none of the 
> alternatives seem particularly attractive to me. I particularly dislike 
> the idea of locking into a single siloed forum service like the 
> bitcointalk forums. I realize I may be in the minority of course.
> 
> 
> Nostr enables the ML team to outsource all of its biggest burdens, if it 
> chooses:
> 
> - mail server blocking is N/A to nostr
> 
> - Hosting costs are completely outsourced unless the ML team chooses to 
> host a relay.
> 
> - Archives and web portal access can be similarly outsourced because any 
> nostr archive is self-authenticating.
> 
> - The ML team can also choose to completely outsource moderation, as 
> nostr is (more or less) permissionless by nature.
>    I understand if there is a "blessed" communication system, the ML 
> team would probably prefer to keep it high quality. To that end there 
> are proposals for proof-of-work, and web of trust based blocklists for 
> nostr which are optional for end users. There are other options such as 
> the "moderated communities" proposal which would provide tighter control.
> 
> 
> On the user side, the optional moderation is very attractive, allowing 
> controversial threads to exist and continue, without requiring everyone 
> to see them.
> 
> 
> The following do not currently exist (to my knowledge) and would need to 
> be implemented to meet the ML's requirements:
> 
> - an email gateway to satisfy the bulk of existing ML subscribers
>    This reintroduces issues with mail server blocking of course.
> - a long-form oriented nostr client (current plain text clients could be 
> used in the meantime)
> 
> That admittedly is quite a lot of work, but the second item can be 
> deferred, and the first is not particularly technically challenging, the 
> complications are all on the administration side.
> 
> I expect some reflexive NACKs based on the immaturity of the ecosystem 
> but if we have months to prepare, I believe the core requirements can be 
> solidly satisfied in time, the rest can be developed over time, and I 
> believe the advantages are worth careful consideration.
> 
> Cheers,
> Dan
> 
> On Tue, Nov 7, 2023 at 9:38 AM Bryan Bishop via bitcoin-dev 
>  > wrote:
> 
> Hello,
> 
> We would like to request community feedback and proposals on the
> future of the mailing list.
> 
> Our current mailing list host, Linux Foundation, has indicated for
> years that they have wanted to stop hosting mailing lists, which
> would mean the bitcoin-dev mailing list would need to move somewhere
> else. We temporarily avoided that, but recently LF has informed a
> moderator that they will cease hosting any mailing lists later this
> year.
> 
> In this email, we will go over some of the history, options, 

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Andrew Chow via bitcoin-dev]

Thanks for writing this up.

I would prefer that we continue to have a mailing list where email is a 
functional and first class user interface. So that would be to migrate 
to groups.io or Google Groups. I think Google Groups is probably the 
better choice of the two.

Although there are concerns that Google would shut down Google Groups or 
specifically target a bitcoin-dev group, I think both are unlikely to 
happen. Both Chromium and Android use Google Groups for their mailing 
lists, so unless those go somewhere else, I doubt Google would 
unceremoniously kill Google Groups. As for shutting down a bitcoin-dev 
group specifically, given that there appears to be several thousand 
groups whose sole purpose is to distribute spam, I don't think Google is 
in the habit of shutting down groups.

Regardless of what we do, there's always the risk that someone will shut 
down or stop maintaining the servers for any discussion medium. Even 
self hosting requires someone to keep the servers up and do maintenance, 
and that person (or people) could get bored of it, run out of money, get 
hit by a bus, etc. We're experiencing that right now with the Linux 
Foundation, and I don't think fear of that should prevent us from moving 
to a different third party host.


Andrew Chow

On 11/07/2023 10:37 AM, Bryan Bishop via bitcoin-dev wrote:
> Hello,
> 
> We would like to request community feedback and proposals on the future 
> of the mailing list.
> 
> Our current mailing list host, Linux Foundation, has indicated for years 
> that they have wanted to stop hosting mailing lists, which would mean 
> the bitcoin-dev mailing list would need to move somewhere else. We 
> temporarily avoided that, but recently LF has informed a moderator that 
> they will cease hosting any mailing lists later this year.
> 
> In this email, we will go over some of the history, options, and invite 
> discussion ahead of the cutoff. We have some ideas but want to solicit 
> feedback and proposals.
> 
> Background
> ==
> 
> The bitcoin-dev mailing list was originally hosted on Sourceforge.net. 
> The bitcoin development mailing list has been a source of proposals, 
> analysis, and developer discussion for many years in the bitcoin 
> community, with many thousands of participants. Later, the mailing list 
> was migrated to the Linux Foundation, and after that OSUOSL began to help.
> 
> Linux Foundation first asked us to move the mailing list in 2017. They 
> internally attempted to migrate all LF mailing lists from mailman2 to 
> mailman3, but ultimately gave up. There were reports of scalability 
> issues with mailman3 for large email communities. Ours definitely 
> qualifies as.. large.
> 
> 2019 migration plan: LF was to turn off mailman and all lists would 
> migrate to the paid service provider groups.io . Back 
> then we were given accounts to try the groups.io  
> interface and administration features. Apparently we were not the only 
> dev community who resisted change. To our surprise LF gave us several 
> years of reprieve by instead handing the subdomain and server-side data 
> to the non-profit OSUOSL lab who instead operated mailman2 for the past 
> ~4 years.
> 
> OSUOSL has for decades been well known for providing server 
> infrastructure for Linux and Open Source development so they were a good 
> fit. This however became an added maintenance burden for the small 
> non-profit with limited resources. Several members of the Bitcoin dev 
> community contributed funding to the lab in support of their Open Source 
> development infrastructure goals. But throwing money at the problem 
> isn’t going to fix the ongoing maintenance burden created by antiquated 
> limitations of mailman2.
> 
> Permalinks
> ==
> 
> Linux Foundation has either offered or agreed to maintain archive 
> permalinks so that content of historic importance is not lost. 
> Fortunately for us while lists.linuxfoundation.org 
>  mailman will go down, they have 
> agreed the read-only pipermail archives will remain online. So all old 
> URLs will continue to remain valid. However, the moderators strongly 
> advise that the community supplements with public-inbox instances to 
> have canonical archive urls that are separate from any particular email 
> software host.
> 
> Public-Inbox
> 
> 
> https://public-inbox.org/README.html 
> 
> “Public Inbox” decentralized archiving - no matter what mailing list 
> server solution is used, anyone can use git to maintain their own 
> mailing list archive and make it available to read on the web.
> 
> Public Inbox is a tool that you can run yourself. You can transform your 
> mbox file and it makes it browsable and viewable online. It commits 
> every post to a git repository. It's kind of like a decentralized mail 
> archiving tool. Anyone can publish the mail archive to any web server 
> they 

Re: [bitcoin-dev] Proposed BIP for MuSig2 PSBT Fields

[Andrew Chow via bitcoin-dev]

On 10/11/2023 07:47 PM, Anthony Towns wrote:
> On Tue, Oct 10, 2023 at 10:28:37PM +0000, Andrew Chow via bitcoin-dev wrote:
>> I've written up a BIP draft for MuSig2 PSBT fields. It can be viewed at
>> https://github.com/achow101/bips/blob/musig2-psbt/bip-musig2-psbt.mediawiki.
> 
> I was hoping to see adaptor signature support in this; but it seems that's
> also missing from BIP 327?

This is the first time I've heard of that, so it wasn't something that I 
considered adding to the BIP. Really the goal was to just be able to use 
BIP 327.

But that doesn't preclude a future BIP that specifies how to use adaptor 
signatures and to have additional PSBT fields for it. It doesn't look 
like those are mutually exclusive in any way or that the fields that 
I've proposed wouldn't still work.

I don't know enough about the topic to really say much on whether or how 
such fields would work.

> FWIW, "participant" is typoed a bunch ("particpant")and the tables are
> hard to read: you might consider putting the description as a separate
> row? eg:
> 
>   https://github.com/ajtowns/bips/blob/202310-table/bip-musig2-psbt.mediawiki

Yes, I've been making some minor fixes throughout the day, and I'll 
probably take your suggestion for the tables. Mediawiki tables are the 
bane of my existence.

Andrew

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Proposed BIP for MuSig2 Descriptors

[Andrew Chow via bitcoin-dev]

Hi All,

I've written up a BIP draft for MuSig2 descriptors. It can be viewed at 
https://github.com/achow101/bips/blob/musig2-descriptors/bip-musig2-descriptors.mediawiki.

Andrew

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Proposed BIP for MuSig2 PSBT Fields

[Andrew Chow via bitcoin-dev]

Hi All,

I've written up a BIP draft for MuSig2 PSBT fields. It can be viewed at 
https://github.com/achow101/bips/blob/musig2-psbt/bip-musig2-psbt.mediawiki.

This is based on this gist from Sanket: 
https://gist.github.com/sanket1729/4b525c6049f4d9e034d27368c49f28a6. 
There are a few notable differences:
- The participant pubkeys field is keyed by only the aggregate xonly key
- Participant pubkeys are compressed pubkeys rather than xonly.

Andrew

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Denial of Service using Package Relay

[Andrew Chow via bitcoin-dev]

On 07/06/2023 12:22 PM, alicexbt via bitcoin-dev wrote:
 > 1) Register input in A
 > 2) Double spend same input with zero fee to your own address
 > 3) Register unconfirmed UTXO from 2 in B

Why would unconfirmed inputs be accepted in a coinjoin? That seems 
unsafe, regardless of package relay. The sender of the unconfirmed 
transaction can already replace it thereby pinning or otherwise 
invalidating the coinjoin, it doesn't need package relay.

Furthermore, the coordinator B shouldn't accept the unconfirmed UTXO 
from 2 because it doesn't even know about that unconfirmed transaction. 
It has zero fee, so it's not going to be relayed.

Conceivably a similar attack can already be done by simply registering 
the same UTXO with multiple coordinators anyways. This doesn't require 
package relay either.

***

Package relay should help coinjoins since any one of the participants 
can rebroadcast the coinjoin with a further CPFP if the coinjoin is 
below the minimum relay fee. Some of the upcoming package RBF proposals 
should also help by allowing other child transactions in the package to 
RBF the entire thing, thereby resolving the need to have everyone 
re-sign the coinjoin in order to RBF.


Andrew

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Bitcoin Core maintainers and communication on merge decisions

[Andrew Chow via bitcoin-dev]

On 05/07/23 03:03 AM, Michael Folkson via bitcoin-dev wrote:

> The decision process for adding a new maintainer was according to the IRC 
> meeting that the maintainers decided privately there was a need for a 
> maintainer “who understood our interfaces and modularization efforts well” 
> and that ryanofsky was a “good fit for that”. I don’t know whether this was 
> decided in a private IRC channel or was decided at the recent in person Core 
> Dev meeting. Regardless, many have had no input into the discussion on what 
> kind of maintainer the project needs going forward and it seems the 
> maintainers do not want to discuss that aspect of the decision.

Since the project began, the decision to seek out and then add a maintainer has 
always been made by existing maintainers. When the maintainers feel that there 
is a need for additional maintainers, they may have an open call for 
volunteers, or may have a candidate already in mind and suggest that specific 
person for maintainership. Contributors generally are not consulted in the 
decision to seek a new maintainer as they would not know whether there are 
things that are being overlooked or that there is maintainership load that 
needs to be distributed. Even so, it wouldn't be appropriate to add a 
maintainer if many contributors disagreed with it, just as with any other PR.

We can take a look at how previous maintainers were added to see how this has 
played out in the past. I think our modern concept of maintainers with nominal 
scopes began in 2015 with Jonas Schnelli. Both Jonas Schnelli and Marco Falke 
were simply announced by Wladimir. There was no public discussion, and some IRC 
logs refer to private emails between the them and the current maintainers at 
that time. After that, meshcollider was added as a maintainer after a public 
"call for maintainers" where a recurring topic for a while was finding a 
maintainer for the wallet. He had volunteered to do it by contacting Wladimir 
privately before it was discussed during an IRC meeting and then on Github. 
Fanquake was added as a maintainer during a CoreDev event in Amsterdam during a 
discussion initiated and led by the maintainers. This was also "private" 
insofar as the discussion was limited to those in attendance, although there 
was some opportunity for public discussion in the PR opened on Github. For 
myself, it was also initially private as I messaged Wladimir to volunteer for 
it after meshcollider stepped down. There was some discussion on IRC and on 
Github, but it was also obvious that many already expected me to be the wallet 
maintainer after meshcollider. Hebasto was added with basically no fanfare or 
discussion - the only mention I can find is the PR itself. My understanding is 
that the maintainers asked him he wanted to do it before the PR was opened. 
Glozow was nominated to be a maintainer by some of the current maintainers, and 
her nomination was really the first time that there was significant public 
discussion about it.

Of the past 7 maintainer additions, 5 were nominations/announcements from the 
current maintainers, one was volunteering following an actual "call for 
maintainer", and one was an obvious successor. It's obvious and common sense 
that the maintainers decide when they need help shouldering the load, and then 
find somebody to help them. There was and always will be some level of private 
communication prior to any public announcement of the nomination or 
volunteering of a maintainer. It doesn't make sense to blindside somebody with 
a nomination without talking to them beforehand. The fact that most of these 
were non-controversial speaks to how well the maintainers were considering 
their nominations before publicly announcing them.

It's also clear that we have been moving towards more open discussion about 
maintainership and who should be maintainers. The process is fundamentally more 
public than it was previously. We now have public discussion with contributors 
about the merits of a person, even if that results in said person not becoming 
a maintainer. Over time, there's been more public participation in the PRs and 
on IRC meetings when maintainer nominations are brought up. We have nominations 
as topics during meetings now when they occur. The PRs to add keys are left 
open for longer to get more discussion.

Ultimately, if you disagree with how the project operates, then you are free to 
leave and start your own fork that is run in a way that you think is 
appropriate. This is open source software, no one is beholden to you, and no 
one is required to do anything.

***

Since you are intent on discussing and re-litigating the decision about Vasil, 
I will agree that we (the maintainers) could have done a better job of 
communicating. However we stand by the decision that was made in the end, and 
we did have a chat with him about it during CoreDev.

It really boils down to three things: 1) we did not ask for a P2P maintainer, 
2) 

Re: [bitcoin-dev] Bitcoin Core maintainers and communication on merge decisions

[Andrew Chow via bitcoin-dev]

Responses in-line.
Note that the opinions expressed in this email are my own and are not
representative of what other maintainers think or believe.

On 04/18/2023 08:40 AM, Michael Folkson via bitcoin-dev wrote:
 >
 > Communication has been a challenge on Bitcoin Core for what I can
tell the entire history of the project. Maintainers merge a pull request
and provide no commentary on why they’ve merged it.

What commentary does there need to be?
It's self evident that the maintainer believes the code is ready to be
merged, and has observed enough ACKs from contributors that they are
comfortable to do so.
You're welcome to ask for clarification, but frankly, I don't think
having any commentary on merges is going to be helpful or more elaborate
in any way.
Requiring maintainers to have to write explanations for every single
merge is simply going to increase the burden on them and increase the
rate of burnout and resignations.
We've had too many maintainers step down already.
It'll end up being a bunch of boilerplate comments that don't say
anything meaningful.

There are certainly situations where PRs are merged very quickly or with
otherwise little apparent review.
But, as I said, if you ask a maintainer why it was merged, the answer
will be "I thought it was ready and had enough review".
There may be other reasons that made the maintainer think it was ready
sooner, such as the PR fixes a critical bug or security vulnerability,
but these reasons aren't going to be stated publicly.

 > Maintainers leave a pull request with many ACKs and few (if any)
NACKs for months and provide no commentary on why they haven't merged it.

There are currently 320 open PRs and 366 open issues.
I wake up every morning to 150+ email notifications containing
everything that went on overnight, and throughout the day, I typically
get hundreds more.
It's impossible to keep up with everything that goes on throughout the repo.
ACKs come in sporadically, PRs are updated, reviews are posted, etc.
Often times PRs are not merged simply because the maintainers were not
aware that a PR was ready to be merged.
Things can simply fall through the cracks.

Of course there are other reasons why something might not be merged, and
these generally fall into the camp of "I don't think it has had enough
review".
It's the maintainer's judgement call to make as to whether something has
been sufficiently reviewed, and part of the judgement call is to
consider the quality and competence of the reviewers.
If a PR had 100 ACKs but all from random people who have never
contributed to the project in any capacity, then it's not going to be
merged because those reviewers would be considered low quality.
It's not just about the numbers, but also about whether the reviewers
are people the maintainers think are familiar enough with an area and
have had a history of thoroughly reviewing PRs.
For example, if a reviewer who primarily works on the mempool reviewed a
PR in the wallet, I would consider their review and ACK with less weight
because they are unlikely to be familiar with the intricacies of the wallet.
Obviously that changes over time as they make more reviews.
For another example, if I see an ACK from a reviewer who posts reviews
that primarily contain nits on code style and other trivialities, I
would consider that ACK with less weight.

Furthermore, the maintainers are not necessarily the ones who block a merge.
Part of evaluating if something is ready to be merged is to read the
comments on a PR.
Other frequent contributors may have commented or asked questions that
haven't been resolved yet.
PRs will often not be merged (even if they have ACKs) until a maintainer
deems that those comments and questions have been sufficiently resolved,
typically with the commenter stating in some way that their concerns
were addressed.
In these situations, no commentary from maintainers is given nor
necessary as it should be self evident (by reading the comments) that
something is controversial.
These kinds of comments are not explicit NACKs (so someone who is only
counting (N)ACKs won't see them), but are blocking nonetheless.

Lastly, personally I like to review every PR before I merge it.
This often means that a PR that might otherwise be ready to be merged
wouldn't be merged by myself as I may not be familiar with that part of
the codebase.
It may also mean that I would require more or specific additional people
to review a PR before I merge it as I would weight my own review less
heavily.
With several long time maintainers stepping away, this may be a factor
in PRs taking longer to get merged as the remaining maintainers may be
less familiar with the parts of the codebase that were previously
maintained by someone else.

 > but a casual observer would have only seen Concept ACKs and ACKs with
3 stray NACKs. Many of these casual observers inflated the numbers on
the utxos.org site [4] signalling support for a soft fork activation
attempt.

Anyone who thinks that 

Re: [bitcoin-dev] OP_VAULT: a new vault proposal

[Andrew Chow via bitcoin-dev]

Hi James,

This seems like a promising proposal, but I noticed have a few issues
regarding batching and privacy.

It seems like this proposal will encourage address reuse for vaults, at
least in some parts. It seems like it would not be difficult to ensure
that each vault address was unique through the use of key derivation.
The recovery and unvault scripts could be produced from ranged
descriptors and so there would each vault address would be unique as
each recovery and unvault script is unique. It would not be hard to have
descriptors for vaults, which would then allow for usage of other
descriptors and miniscript into the recovery and unvault scripts.

However the current construction makes it impossible to spend these
vaults together. Since OP_VAULT requires the recovery script of the
unvault output to match what's provided in the input, if there are
multiple inputs with different recovery scripts, then the transaction
will fail. I'm not sure how this could be solved though.

But from my reading of the code, it looks like the unvault scripts can
be unique, so at least address reuse can be avoided here. It just means
that the recovery scripts must be the same, and this would leave an
identifying mark on chain for every unvault. An observer would be able
to correlate unvault transactions by the hashes of the recovery scripts,
and I think this would be rather detrimental to user privacy, not to
mention that sweeping to recovery would also reveal all of your coins too.

On the topic of address reuse, the implemented optional re-vault output
explicitly requires address reuse, as well as breaking the batched
unvaulting of scripts that have different unvault scripts. It's
currently implemented as requiring the unvault script to exactly match
the prevout script of the inputs being spent. This means that all inputs
must have the same script. I think it would be sufficient to do the same
check as the OP_UNVAULT script and just require that the recovery script
and the delay are the same, with the hash of the trigger script being
provided in the input in the same way the target hash is provided for
OP_UNVAULT. This would break the address reuse requirement.

I'm also not convinced that OP_VAULT and OP_UNVAULT should be allowed
for bare and P2WSH outputs. It seems like it would make sense to just
limit their usage to tapscripts as this would simply their implementation.


Andrew

On 01/09/2023 11:07 AM, James O'Beirne via bitcoin-dev wrote:
> For the last few years, I've been interested in vaults as a way to
> substantially derisk custodying Bitcoin, both at personal and commercial
> scales. Instead of abating with familiarity, as enthusiasm sometimes
> does, my conviction that vaults are an almost necessary part of bitcoin's
> viability has only grown over the years.
>
> Since people first started discussing vaults, it's been pretty clear that
> some kind of covenant-enabling consensus functionality is necessary to
> provide the feature set necessary to make vault use practical.
>
> Earlier last year I experimented with using OP_CTV[1], a limited covenant
> mechanism, to implement a "minimum-viable" vault design. I found that the
> inherent limitations of a precomputed covenant scheme left the resulting
> vault implementation wanting, even though it was an improvement over
> existing strategies that rely on presigned transactions and (hopefully)
> ephemeral keys.
>
> But I also found proposed "general" covenant schemes to be
> unsuitable for this use. The bloated scriptPubKeys, both in size and
> complexity, that would result when implementing something like a vault
> weren't encouraging. Also importantly, the social-consensus quagmire
> regarding which covenant proposal to actually deploy feels at times
> intractable.
>
> As a result, I wanted to explore a middle way: a design solely concerned
> with making the best vault use possible, with covenant functionality as a
> secondary consideration. In other words, a proposal that would deliver
> the safety benefits of vaults to users without getting hung up on
> trying to solve the general problem of covenants.
>
> At first this design, OP_VAULT, was just sort of a pipe dream. But as I
> did more thinking (and eventually implementing) I became more convinced
> that, even if it isn't considered for soft-fork, it is a worthwhile
> device to serve as a standard benchmark against which other proposals
> might be judged.
>
> I wrote a paper that summarizes my findings and the resulting proposal:
> https://jameso.be/vaults.pdf
>
> along with an accompanying draft implementation:
> https://github.com/bitcoin/bitcoin/pull/26857
>
> I might work on a BIP if there's interest.
>
> James
>
> [1]: https://github.com/jamesob/simple-ctv-vault


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Huge wallets make Bitcoin Core unusable (Daemon+CLI & Qt)

[Andrew Chow via bitcoin-dev]

This is a known issue that I've been working on. The wallet is a large module 
in Bitcoin Core and changing it takes quite a bit of time.

On 08/20/2022 10:16 AM, Felipe Micaroni Lalli via bitcoin-dev wrote:

> 8.1) Can we "optimize" a huge wallet without moving the funds to a new one? 
> Like a "fsck" or eqv?

You can remove old transactions using the removeprunedfunds RPC. That should 
greatly speed up balance calculations and transaction creation.

> 8.2) Can we improve the cache usage somehow? Putting the entire wallet in 
> memory, for example?

It's already entirely in memory.___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposal: Receiving and Change Derivation Paths in a Single Descriptor

[Andrew Chow via bitcoin-dev]

I've updated the BIP text to allow arbitrary length tuples.

On 07/27/2022 04:44 AM, Pavol Rusnak wrote:

> On Wed, 27 Jul 2022 at 00:28, Andrew Chow  wrote:
>
>> However I don't see why this couldn't generalize to any sized tuples. As 
>> long as the tuples are all the same length, and the limit is one tuple per 
>> key expression, then we don't get any combinatorial blowup issues.
>
> I think it's worthwhile to generalize for any sized tuples. I don't have any 
> existing particular use case in mind, because BIP-44, BIP-84, etc. are fine 
> with just using <0;1>, but there might be some upcoming standards in the 
> future that will want to introduce more sub-paths.
>
> --
>
> Best Regards / S pozdravom,
>
> Pavol "stick" Rusnak
> Co-Founder, SatoshiLabs___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposal: Receiving and Change Derivation Paths in a Single Descriptor

[Andrew Chow via bitcoin-dev]

I went with just tuples of two values because that's easier to implement and 
targets exactly what people were asking for. However I don't see why this 
couldn't generalize to any sized tuples. As long as the tuples are all the same 
length, and the limit is one tuple per key expression, then we don't get any 
combinatorial blowup issues.

Are there any use cases for variable length tuples?

Andrew

On 07/26/2022 05:56 PM, Pavol Rusnak wrote:

> Thanks Andrew for this BIP. We've been already using this for quite some time 
> for Trezor in production.
>
> Just one clarification: Should , , ... also 
> work or we only aim to support only tuples of exactly two values?
>
> On Tue, 26 Jul 2022 at 23:51, Andrew Chow via bitcoin-dev 
>  wrote:
>
>> Hi All,
>>
>> I would like to propose a BIP that de-duplicates and simplifies how we
>> represent descriptors for receiving and change addresses. Under the
>> existing BIPs, this requires two descriptors, where the vast majority of
>> the descriptors are the same, except for a single derivation path
>> element. This proposal allows descriptors to have a single derivation
>> path element that can specify a pair of indexes. Parsers would then
>> expand these into two almost identical descriptors with the difference
>> being that the first uses the first of the pair of indexes, and the
>> second uses the second.
>>
>> The proposed notation is ``. As an example,
>> `wpkh(xpub.../0/<0;1>/*)` would be expanded into `wpkh(xpub.../0/0/*)`
>> and `wpkh(xpub.../0/1/*)`.
>>
>> This also works for descriptors involving multiple keys - the first
>> element in every pair is used for the first descriptor, and the second
>> element of each pair in the second descriptor.
>>
>> The full text of the BIP can be found at
>> https://github.com/achow101/bips/blob/bip-multipath-descs/bip-multipath-descs.mediawiki
>> and also copied below. An implementation of it to Bitcoin Core is
>> available at https://github.com/bitcoin/bitcoin/pull/22838.
>>
>> Any feedback on this would be appreciated.
>>
>> Thanks,
>> Andrew Chow
>>
>> ---
>>
>> 
>> BIP: multipath-descs
>> Layer: Applications
>> Title: Multipath Descriptor Key Expressions
>> Author: Andrew Chow 
>> Comments-Summary: No comments yet.
>> Comments-URI:
>> https://github.com/bitcoin/bips/wiki/Comments:BIP-multipath-descs
>> Status: Draft
>> Type: Informational
>> Created: 2022-07-26
>> License: BSD-2-Clause
>> 
>>
>> ==Abstract==
>>
>> This document specifies a modification to Key Expressions of Descriptors
>> that are described in BIP 380.
>> This modification allows Key Expressions to indicate BIP 32 derivation
>> path steps that can have multiple values.
>>
>> ==Copyright==
>>
>> This BIP is licensed under the BSD 2-clause license.
>>
>> ==Motivation==
>>
>> Descriptors can describe the scripts that are used in a wallet, but
>> wallets often require at least two descriptors for all of the scripts
>> that they watch for.
>> Wallets typically have one descriptor for producing receiving addresses,
>> and the other for change addresses.
>> These descriptors are often extremely similar - they produce the same
>> types of scripts, derive keys from the same master key, and use
>> derivation paths that are almost identical.
>> The only differences are in the derivation path where one of the steps
>> will be different between the descriptors.
>> Thus it is useful to have a notation to represent both descriptors as a
>> single descriptor where one of the derivation steps is a pair of values.
>>
>> ==Specification==
>>
>> For extended keys and their derivations paths in a Key Expression, BIP
>> 380 states:
>>
>> * xpub encoded extended public key or xprv encoded
>> extended private key (as defined in BIP 32)
>> ** Followed by zero or more /NUM or /NUMh path
>> elements indicating BIP 32 derivation steps to be taken after the given
>> extended key.
>> ** Optionally followed by a single /* or /*h final
>> step to denote all direct unhardened or hardened children.
>>
>> This is modifed to state:
>>
>> * xpub encoded extended public key or xprv encoded
>> extended private key (as defined in BIP 32)
>> ** Followed by zero or more /NUM or /NUMh path
>> elements indicating BIP 32 derivation steps to be taken after the given
>> extended key.
>> ** Followed by zero or one / (NUM may be
>> followed by h to indicated a hardened step) path element
>> indicating a 

[bitcoin-dev] BIP Proposal: Receiving and Change Derivation Paths in a Single Descriptor

[Andrew Chow via bitcoin-dev]

Hi All,

I would like to propose a BIP that de-duplicates and simplifies how we
represent descriptors for receiving and change addresses. Under the
existing BIPs, this requires two descriptors, where the vast majority of
the descriptors are the same, except for a single derivation path
element. This proposal allows descriptors to have a single derivation
path element that can specify a pair of indexes. Parsers would then
expand these into two almost identical descriptors with the difference
being that the first uses the first of the pair of indexes, and the
second uses the second.

The proposed notation is ``. As an example,
`wpkh(xpub.../0/<0;1>/*)` would be expanded into `wpkh(xpub.../0/0/*)`
and `wpkh(xpub.../0/1/*)`.

This also works for descriptors involving multiple keys - the first
element in every pair is used for the first descriptor, and the second
element of each pair in the second descriptor.

The full text of the BIP can be found at
https://github.com/achow101/bips/blob/bip-multipath-descs/bip-multipath-descs.mediawiki
and also copied below. An implementation of it to Bitcoin Core is
available at https://github.com/bitcoin/bitcoin/pull/22838.

Any feedback on this would be appreciated.

Thanks,
Andrew Chow

---


   BIP: multipath-descs
   Layer: Applications
   Title: Multipath Descriptor Key Expressions
   Author: Andrew Chow 
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-multipath-descs
   Status: Draft
   Type: Informational
   Created: 2022-07-26
   License: BSD-2-Clause


==Abstract==

This document specifies a modification to Key Expressions of Descriptors
that are described in BIP 380.
This modification allows Key Expressions to indicate BIP 32 derivation
path steps that can have multiple values.

==Copyright==

This BIP is licensed under the BSD 2-clause license.

==Motivation==

Descriptors can describe the scripts that are used in a wallet, but
wallets often require at least two descriptors for all of the scripts
that they watch for.
Wallets typically have one descriptor for producing receiving addresses,
and the other for change addresses.
These descriptors are often extremely similar - they produce the same
types of scripts, derive keys from the same master key, and use
derivation paths that are almost identical.
The only differences are in the derivation path where one of the steps
will be different between the descriptors.
Thus it is useful to have a notation to represent both descriptors as a
single descriptor where one of the derivation steps is a pair of values.

==Specification==

For extended keys and their derivations paths in a Key Expression, BIP
380 states:

* xpub encoded extended public key or xprv encoded
extended private key (as defined in BIP 32)
** Followed by zero or more /NUM or /NUMh path
elements indicating BIP 32 derivation steps to be taken after the given
extended key.
** Optionally followed by a single /* or /*h final
step to denote all direct unhardened or hardened children.

This is modifed to state:

* xpub encoded extended public key or xprv encoded
extended private key (as defined in BIP 32)
** Followed by zero or more /NUM or /NUMh path
elements indicating BIP 32 derivation steps to be taken after the given
extended key.
** Followed by zero or one / (NUM may be
followed by h to indicated a hardened step)  path element
indicating a pair of BIP 32 derivation steps to be taken after the given
extended key.
** Followed by zero or more /NUM or /NUMh path
elements indicating BIP 32 derivation steps to be taken after the given
extended key.
** Optionally followed by a single /* or /*h final
step to denote all direct unhardened or hardened children.

When a / is encountered, parsers should produce two
descriptors where the first descriptor uses the first NUM, and
a second descriptor uses the second NUM.

The common use case for this is to represent descriptors for producing
receiving and change addresses.
When interpreting for this use case, wallets should use the first
descriptor for producing receiving addresses, and the second descriptor
for producing change addresses.
For this use case, the element will commonly be the value /<0;1>

==Test Vectors==

TBD

==Backwards Compatibility==

This is an addition to the Key Expressions defined in BIP 380.
Key Expressions using the format described in BIP 380 are compatible
with this modification and parsers that implement this will still be
able to parse such descriptors.
However as this is an addition to Key Expressions, older parsers will
not be able to understand such descriptors.

This modification to Key Expressions uses two new characters: <
and ;.
These are part of the descriptor character set and so are covered by the
checksum algorithm.
As these are previously unused characters, old parsers will not
accidentally mistake them for indicating something else.

==Reference Implementation==

https://github.com/bitcoin/bitcoin/pull/22838

Re: [bitcoin-dev] bitcoinj fork with Taproot support

[Andrew Chow via bitcoin-dev]

Prior to 0.19.0, creating outputs with an unknown witness version was 
considered non-standard. This was a violation of BIP 173 and was fixed for 
0.19.0+ in PR #15846.

On 11/16/2021 10:17 PM, n1ms0s via bitcoin-dev wrote:

> Hello all,
> I am currently working on a fork of bitcoinj with basic Taproot support. 
> Currently it supports basic sending and receiving with Taproot addresses 
> using a bitcoinj SPV wallet.
> See here: https://github.com/n1ms0s/bitcoinj
>
> It supports the above along with public/private key tweaking. Feel free to 
> take a look, and leave feedback or even work on it yourself and submit a pull 
> request.
>
> One issue I am running into right now though is when broadcasting a Taproot 
> transaction to older nodes (old as in ~0.18.0) I get an error response of 
> "Witness version reserved for soft-fork upgrades". Anyone have any idea why 
> this happens? I have a stackexchange question open here for it:
> https://bitcoin.stackexchange.com/questions/110787/issue-when-broadcasting-taproot-transaction-to-older-nodes
>
> n1ms0s___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] bitcoin.org missing bitcoin core version 22.0

[Andrew Chow via bitcoin-dev]

On 10/20/2021 03:20 PM, Owen Gunden via bitcoin-dev wrote:
> I also notice that, as of 22.0, Wladimir is no longer signing the
> releases, and I have no trust in my gpg network of the people who seem
> to have replaced him.
It is signed with his personal key, as well as the personal keys of
several other developers.

> Given the level of security at stake here, my eyebrows are raised at
> this combination of items changing (new website + new gpg signers at the
> same time).
bitcoincore.org has been Bitcoin Core's official website for several
years. Binaries have not been posted to bitcoin.org by the release
maintainer for several major releases. The only reason they are still
available there is because bitcoin.org's maintainer mirrors them.

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Proposal for a few IANA mime-types related to Bitcoin

[Andrew Chow via bitcoin-dev]

Hi Peter,

It would be nice to have mime types registered for Bitcoin things, but
I'm not sure that it will be possible, at least not in the way that we
would like. I tried doing this with "application/bitcoin-psbt" back in
2019 but it was not accepted. From that attempt, here is what I have
learned:

There are only a few accepted top level types, so we would not be able
to use "bitcoin" as the top level (unless you want to submit an RFC to
add a "bitcoin" top level). Of the available top level types,
"application" is the most appropriate for Bitcoin.

Next is the tree that the mime type should be in. The best would be the
Standards tree, but it has some requirements that Bitcoin doesn't really
meet. In order to be in the standards tree, the registration must be
either associated with an IETF specification (so a RFC) or registered by
a recognized standards related organization. Unfortunately the closest
thing to a standards organization that Bitcoin has is the BIPs process,
and that is not a really a standards organization nor is it recognized
by IANA. So in order to register the mimetypes as Standards tree types,
we would need to write an RFC, but this could be an independent
submission (https://www.rfc-editor.org/about/independent/) rather than
IETF-stream submission. I did not continue to pursue this because I
didn't have the time.

Another alternative would be to use the Vendor tree, but that would
prefix the mimetype with "vnd." so it would end up being something like
"application/vnd.bitcoin.psbt". I did not think this was an reasonable
so I did not continue to pursue this avenue.


Andrew Chow

On 8/31/21 2:27 PM, Peter D. Gray via bitcoin-dev wrote:
> Hi list!
>
> I am proposing to register the following MIME (RFC 2046) media types with the 
> IANA:
>
>
> bitcoin/psbt
>
>  - aka. a BIP-174 file, in binary
>  - does not make any claims about signed/unsigned status; lets leave that 
> to the file
>
> bitcoin/txn
>
>  - aka. wire-ready fully-signed transaction in binary
>
> bitcoin/uri
>
>  - aka 
> [BIP-21](https://github.com/bitcoin/bips/blob/master/bip-0021.mediawiki)
>  - could be just a bare bech32 or base58 payment address
>  - but can also encode amount, comments in URL args
>  - potentially interesting as a response to 402 - Payment required
>
>
> Other thoughts
>
> - some mime-types are proposed in BIP-71 but those are unrelated to above, 
> and never
>seem to have been registered
>
> - for those who like to encode their binary as base64 or hex, that can be 
> indicated
>as "encoding=hex" or "encoding=base64" in the optional parameters, just 
> like
>"text/plain; encoding=utf-8" does. However, the default must be binary.
>
> - although the above are useful for web servers, they are also useful 
> elsewhere and I
>intend to use them in NFC (NDEF records) where a shorter length is 
> critical.
>
> - I have no idea how easily IANA will accept these proposals.
>
> - current approved mime types: 
> https://www.iana.org/assignments/media-types/media-types.xhtml
>
> Thoughts?
>
> ---
> @DocHEX  ||  Coinkite  ||  PGP: A3A31BAD 5A2A5B10
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Andrew Chow via bitcoin-dev]

On 7/2/21 11:24 PM, David A. Harding wrote:
> Is there any chance we can take this opportunity to make "h"/"H" the
> preferred aliases?  Using "'" in bourne-style shells is very
> annoying[1], and I suspect it's also creating unnecessary complications
> elsewhere.
I've updated the text to use "h".
> Alternatives:
>
> - Completely kill "'" (I'd prefer this, but I realize it's complicated
>with descriptors already being used widely).  If "h"/"H" are made the
>preferred aliases, maybe it'd be enough to make implementing "'" a
>SHOULD rather than a MUST; this would push implementations towards
>displaying descriptors using the h versions for maximum compatibility.
Since there already are software implementing descriptors, I don't think
we can do this. I'm not sure about making "'" a SHOULD either.
> - Calculate the checksum over s/(h|H)/'/ (again, I know that's
>complicated with descriptors already widely used)
This has been discussed in the past and the conclusion was that the
checksum should be strictly over the string itself. This would allow for
dumb checksum checkers which don't have to be able to parse descriptors
in order to check the checksum.

Thanks,
Andrew

>
> Thanks,
>
> -Dave
>
> [1] https://github.com/bitcoin/bitcoin/issues/15740#issuecomment-695815432


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Andrew Chow via bitcoin-dev]

I've opened a PR against my own BIPs fork for review: 
https://github.com/achow101/bips/pull/3

Andrew

On 6/29/21 11:41 PM, Jeremy wrote:

> Kudos, this is fantastic!
>
> It might be easier, since there is a ton of content here, for you to open up 
> some WIP PRs to collect feedback?
> --
> [@JeremyRubin](https://twitter.com/JeremyRubin)
>
> On Tue, Jun 29, 2021 at 2:15 PM Andrew Chow via bitcoin-dev 
>  wrote:
>
>> Hi All,
>>
>> I've been working on formalizing the Output Script Descriptors that have
>> been available in Bitcoin Core for a while into BIPs. Since descriptors
>> are modular and have optional components, I've decided to split it into
>> 7 BIPs, rather than a single one. The first describes descriptors in
>> general and does not specify any particular descriptor. However it does
>> describe the general operation, key expressions (including derivation
>> paths and key origin info), and the descriptor checksum. The following 6
>> BIPs specify the actual descriptors themselves. These are non-segwit
>> descriptor (pk, pkh, sh), segwit descriptors (wpkh, wsh), multisig
>> descriptors (multi, sortedmulti), the taproot descriptor (tr), the combo
>> descriptor, and opaque descriptors (raw, addr). This separation is so
>> that implementors can choose to not implement some descriptors and still
>> say which descriptors they support without being too difficult to
>> understand.
>>
>> The text of all of the documents are below, and they can also be found
>> on github:https://github.com/achow101/bips/tree/descriptors/
>>
>> Thanks,
>> Andrew Chow
>>
>> ---
>>
>> 
>> BIP: bip-descriptors-general
>> Layer: Applications
>> Title: Output Script Descriptors General Operation
>> Author: Pieter Wuille 
>> Andrew Chow 
>> Comments-Summary: No comments yet.
>> Comments-URI:
>> https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-general
>> Status: Draft
>> Type: Informational
>> Created: 2021-06-27
>> License: BSD-2-Clause
>> 
>>
>> ==Abstract==
>>
>> Output Script Descriptors are a simple language which can be used to
>> describe collections ofoutput scripts.
>> There can be many different descriptor fragments and functions.
>> This document describes the general syntax for descriptors, descriptor
>> checksums, and common expressions.
>>
>> ==Copyright==
>>
>> This BIP is licensed under the BSD 2-clause license.
>>
>> ==Motivation==
>>
>> Bitcoin wallets traditionally have stored a set of keys which are later
>> serialized and mutated to produce the output scripts that the wallet
>> watches and the addresses it provides to users.
>> Typically backups have consisted of solely the private keys, nowadays
>> primarily in the form of BIP 39 mnemonics.
>> However this backup solution is insuffient, especially since the
>> introduction of Segregated Witness which added new output types.
>> Given just the private keys, it is not possible for restored wallets to
>> know which kinds of output scripts and addresses to produce.
>> This has lead to incompatibilities between wallets when restoring a
>> backup or exporting data for a watch only wallet.
>>
>> Further complicating matters are BIP 32 derivation paths.
>> Although BIPs 44, 49, and 84 have specified standard BIP 32 derivation
>> paths for different output scripts and addresses, not all wallets
>> support them nor use those derivation paths.
>> The lack of derivation path information in these backups and exports
>> leads to further incompatibilities between wallets.
>>
>> Current solutions to these issues have not been generic and can be
>> viewed as being layer violations.
>> Solutions such as introducing different version bytes for extended key
>> serialization both are a layer violation (key derivation should be
>> separate from script type meaning) and specific only to a particular
>> derivation path and script type.
>>
>> Output Script Descriptors introduces a generic solution to these issues.
>> Script types are specified explicitly through the use of Script Expressions.
>> Key derivation paths are specified explicitly in Key Expressions.
>> These allow for creating wallet backups and exports which specify the
>> exact scripts, subscripts (redeemScript, witnessScript, etc.), and keys
>> to produce.
>> With the general structure specified in this BIP, new Script Expressions
>> can be introduced as new script types are added.
>> Lastly, the use of common terminology and existing standards a

Re: [bitcoin-dev] Derivation Paths for Single Key Taproot Scripts

[Andrew Chow via bitcoin-dev]

This was assigned BIP number 86, so the purpose level path will be m/86'

Andrew

On 6/22/21 9:17 PM, Andrew Chow wrote:
> Hi All,
>
> I would like to propose a simple derivation path scheme for keys to be
> used in single key Taproot scripts. This is based on BIP 44 so it is
> basically identical to BIPs 49 and 84. Like with those BIPs, the actual
> value to be used in the purpose level will be set to the BIP number,
> once assigned.
>
> Note that the keys derived in this method should be for the Taproot
> internal key, which should then be tweaked with the hash of itself as
> recommended by BIP 341. The keys derived at this path should not be used
> directly as the Taproot output pubkey. Additionally, this BIP does not
> specify new version bytes for extended key serialization because, with
> the advent of descriptors, I think that is unnecessary. In fact, this
> BIP feels somewhat unnecessary to me, but it seems like it will be
> needed for now in order to drive adoption and implementation of Taproot
> into software and hardware wallets.
>
> The text can be viewed below, with the rendered text available at
> https://github.com/achow101/bips/blob/taproot-bip44/bip-taproot-bip44.mediawiki
>
> Andrew Chow
>
> ---
>
> 
>     BIP: bip-taproot-bip44
>     Layer: Applications
>     Title: Derivation scheme for P2TR based accounts
>     Author: Andrew Chow 
>     Comments-Summary: No comments yet.
>     Comments-URI:
> https://github.com/bitcoin/bips/wiki/Comments:BIP-taproot-bip44
>     Status: Draft
>     Type: Informational
>     Created: 2021-06-22
>     License: BSD-2-Clause
> 
>
> ==Abstract==
>
> This document suggests a derivation scheme for HD wallets whose keys are
> involved in single key
> P2TR ([[bip-0341.mediawiki|BIP 341]]) outputs as the Taproot internal key.
>
> ===Copyright===
>
> This BIP is licensed under the 2-clause BSD license.
>
> ==Motivation==
>
> With the usage of single key P2TR transactions, it is useful to have a
> common derivation scheme so
> that HD wallets that only have a backup of the HD seed can be likely to
> recover single key Taproot
> outputs. Although there are now solutions which obviate the need for
> fixed derivation paths for
> specific script types, many software wallets and hardware signers still
> use seed backups which
> lack derivation path and script information. Thus we largely use the
> same approach used in BIPs
> [[bip-0049.mediawiki|49]] and [[bip-0084.mediawiki|84]] for ease of
> implementation.
>
> ==Specifications==
>
> This BIP defines the two needed steps to derive multiple deterministic
> addresses based on a
> [[bip-0032.mediawiki|BIP 32]] master private key.
>
> ===Public key derivation===
>
> To derive a public key from the root account, this BIP uses the same
> account-structure as
> defined in BIPs [[bip-0044.mediawiki|44]], [[bip-0049.mediawiki|49]],
> and [[bip-0084.mediawiki|84]],
> but with a different purpose value for the script type.
>
> 
> m / purpose' / coin_type' / account' / change / address_index
> 
>
> For the purpose-path level it uses '.
> The rest of the levels are used as defined in BIPs 44, 49, and 84.
>
> ===Address derivation===
>
> To derive the output key used in the P2TR script from the derived public
> key, we use the method
> recommended in
> [[bip-0341.mediawiki#constructing-and-spending-taproot-outputs|BIP 341]]:
>
> 
> internal_key:   lift_x(derived_key)
> 32_byte_output_key: internal_key + int(HashTapTweak(bytes(internal_key)))G
> 
>
> In a transaction, the scripts and witnesses are as defined in
> [[bip-0341.mediawiki#specification|BIP 341]]:
>
> 
> witness:  
> scriptSig:    (empty)
> scriptPubKey: 1 <32_byte_output_key>
>     (0x5120{32_byte_output_key})
> 
>
> ==Backwards Compatibility==
>
> This BIP is not backwards compatible by design.
> An incompatible wallet will not discover these accounts at all and the
> user will notice that
> something is wrong.
>
> However this BIP uses the same method used in BIPs 44, 49, and 84, so it
> should not be difficult
> to implement.
>
> ==Test vectors==
>
> TBD
>
> ==Reference==
>
> * [[bip-0032.mediawiki|BIP32 - Hierarchical Deterministic Wallets]]
> * [[bip-0043.mediawiki|BIP43 - Purpose Field for Deterministic Wallets]]
> * [[bip-0044.mediawiki|BIP44 - Multi-Account Hierarchy for Deterministic
> Wallets]]
> * [[bip-0049.mediawiki|BIP49 - Derivation scheme for
> P2WPKH-nested-in-P2SH based accounts]]
> * [[bip-0084.mediawiki|BIP84 - Derivation scheme for P2WPKH based accounts]]
> * [[bip-0341.mediawiki|BIP341 - Taproot: SegWit version 1 spending rules]]
>


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposals for Output Script Descriptors

[Andrew Chow via bitcoin-dev]

On 6/29/21 6:22 PM, Christopher Allen wrote:

> Are there any plans other than `raw` to support time locks in descriptors?
>
> Any plans for descriptors offering closer integration with miniscript?

I expect miniscript to be a followup BIP that extends these descriptors. 
Miniscript has timelock functionality.

Andrew

> All of Blockchain Commons libraries and tools are multisig descriptor 
> centric, and there are many scenarios that require describing time locks:
>
> - [Designing Multisig for Independence & 
> Resilience](https://github.com/BlockchainCommons/Gordian/blob/master/Docs/Multisig.md)
>
> — Christopher Allen___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP Proposals for Output Script Descriptors

[Andrew Chow via bitcoin-dev]

Hi All,

I've been working on formalizing the Output Script Descriptors that have
been available in Bitcoin Core for a while into BIPs. Since descriptors
are modular and have optional components, I've decided to split it into
7 BIPs, rather than a single one. The first describes descriptors in
general and does not specify any particular descriptor. However it does
describe the general operation, key expressions (including derivation
paths and key origin info), and the descriptor checksum. The following 6
BIPs specify the actual descriptors themselves. These are non-segwit
descriptor (pk, pkh, sh), segwit descriptors (wpkh, wsh), multisig
descriptors (multi, sortedmulti), the taproot descriptor (tr), the combo
descriptor, and opaque descriptors (raw, addr). This separation is so
that implementors can choose to not implement some descriptors and still
say which descriptors they support without being too difficult to
understand.

The text of all of the documents are below, and they can also be found
on github:https://github.com/achow101/bips/tree/descriptors/

Thanks,
Andrew Chow

---


   BIP: bip-descriptors-general
   Layer: Applications
   Title: Output Script Descriptors General Operation
   Author: Pieter Wuille 
   Andrew Chow 
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-descriptors-general
   Status: Draft
   Type: Informational
   Created: 2021-06-27
   License: BSD-2-Clause


==Abstract==

Output Script Descriptors are a simple language which can be used to
describe collections ofoutput scripts.
There can be many different descriptor fragments and functions.
This document describes the general syntax for descriptors, descriptor
checksums, and common expressions.

==Copyright==

This BIP is licensed under the BSD 2-clause license.

==Motivation==

Bitcoin wallets traditionally have stored a set of keys which are later
serialized and mutated to produce the output scripts that the wallet
watches and the addresses it provides to users.
Typically backups have consisted of solely the private keys, nowadays
primarily in the form of BIP 39 mnemonics.
However this backup solution is insuffient, especially since the
introduction of Segregated Witness which added new output types.
Given just the private keys, it is not possible for restored wallets to
know which kinds of output scripts and addresses to produce.
This has lead to incompatibilities between wallets when restoring a
backup or exporting data for a watch only wallet.

Further complicating matters are BIP 32 derivation paths.
Although BIPs 44, 49, and 84 have specified standard BIP 32 derivation
paths for different output scripts and addresses, not all wallets
support them nor use those derivation paths.
The lack of derivation path information in these backups and exports
leads to further incompatibilities between wallets.

Current solutions to these issues have not been generic and can be
viewed as being layer violations.
Solutions such as introducing different version bytes for extended key
serialization both are a layer violation (key derivation should be
separate from script type meaning) and specific only to a particular
derivation path and script type.

Output Script Descriptors introduces a generic solution to these issues.
Script types are specified explicitly through the use of Script Expressions.
Key derivation paths are specified explicitly in Key Expressions.
These allow for creating wallet backups and exports which specify the
exact scripts, subscripts (redeemScript, witnessScript, etc.), and keys
to produce.
With the general structure specified in this BIP, new Script Expressions
can be introduced as new script types are added.
Lastly, the use of common terminology and existing standards allow for
Output Script Descriptors to be engineer readable so that the results
can be understood at a glance.

==Specification==

Descriptors consist of several types of expressions.
The top level expression is a SCRIPT.
This expression may be followed by #CHECKSUM, where
CHECKSUM is an 8 character alphanumeric descriptor checksum.

===Script Expressions===

Script Expressions (denoted SCRIPT) are expressions which
correspond directly with a Bitcoin script.
These expressions are written as functions and take arguments.
Such expressions have a script template which is filled with the
arguments correspondingly.
Expressions are written with a human readable identifier string with the
arguments enclosed with parentheses.
The identifier string should be alphanumeric and may include underscores.

The arguments to a script expression are defined by that expression itself.
They could be a script expression, a key expression, or some other
expression entirely.

===Key Expressions===

A common expression used as an argument to script expressions are key
expressions (denoted KEY).
These represent a public or private key and, optionally, information
about the origin of that key.
Key expressions 

Re: [bitcoin-dev] Taproot Fields for PSBT

[Andrew Chow via bitcoin-dev]

Hi Salvatore,

On 6/28/21 6:03 AM, Salvatore Ingala wrote:

> Hi Andrew,
>
> I just have a small suggestion on this proposal.
>
> On Tue, 22 Jun 2021 at 23:29, Andrew Chow via bitcoin-dev 
>  wrote:
>
>> | Taproot Leaf Script
>> | PSBT_IN_TAP_LEAF_SCRIPT = 0x15
>> | 
>> | The control block for this leaf as specified in BIP 341. The control
>> block contains the merkle tree path to this leaf.
>> | 

[bitcoin-dev] Derivation Paths for Single Key Taproot Scripts

[Andrew Chow via bitcoin-dev]

Hi All,

I would like to propose a simple derivation path scheme for keys to be
used in single key Taproot scripts. This is based on BIP 44 so it is
basically identical to BIPs 49 and 84. Like with those BIPs, the actual
value to be used in the purpose level will be set to the BIP number,
once assigned.

Note that the keys derived in this method should be for the Taproot
internal key, which should then be tweaked with the hash of itself as
recommended by BIP 341. The keys derived at this path should not be used
directly as the Taproot output pubkey. Additionally, this BIP does not
specify new version bytes for extended key serialization because, with
the advent of descriptors, I think that is unnecessary. In fact, this
BIP feels somewhat unnecessary to me, but it seems like it will be
needed for now in order to drive adoption and implementation of Taproot
into software and hardware wallets.

The text can be viewed below, with the rendered text available at
https://github.com/achow101/bips/blob/taproot-bip44/bip-taproot-bip44.mediawiki

Andrew Chow

---


   BIP: bip-taproot-bip44
   Layer: Applications
   Title: Derivation scheme for P2TR based accounts
   Author: Andrew Chow 
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-taproot-bip44
   Status: Draft
   Type: Informational
   Created: 2021-06-22
   License: BSD-2-Clause


==Abstract==

This document suggests a derivation scheme for HD wallets whose keys are
involved in single key
P2TR ([[bip-0341.mediawiki|BIP 341]]) outputs as the Taproot internal key.

===Copyright===

This BIP is licensed under the 2-clause BSD license.

==Motivation==

With the usage of single key P2TR transactions, it is useful to have a
common derivation scheme so
that HD wallets that only have a backup of the HD seed can be likely to
recover single key Taproot
outputs. Although there are now solutions which obviate the need for
fixed derivation paths for
specific script types, many software wallets and hardware signers still
use seed backups which
lack derivation path and script information. Thus we largely use the
same approach used in BIPs
[[bip-0049.mediawiki|49]] and [[bip-0084.mediawiki|84]] for ease of
implementation.

==Specifications==

This BIP defines the two needed steps to derive multiple deterministic
addresses based on a
[[bip-0032.mediawiki|BIP 32]] master private key.

===Public key derivation===

To derive a public key from the root account, this BIP uses the same
account-structure as
defined in BIPs [[bip-0044.mediawiki|44]], [[bip-0049.mediawiki|49]],
and [[bip-0084.mediawiki|84]],
but with a different purpose value for the script type.


m / purpose' / coin_type' / account' / change / address_index


For the purpose-path level it uses '.
The rest of the levels are used as defined in BIPs 44, 49, and 84.

===Address derivation===

To derive the output key used in the P2TR script from the derived public
key, we use the method
recommended in
[[bip-0341.mediawiki#constructing-and-spending-taproot-outputs|BIP 341]]:


internal_key:   lift_x(derived_key)
32_byte_output_key: internal_key + int(HashTapTweak(bytes(internal_key)))G


In a transaction, the scripts and witnesses are as defined in
[[bip-0341.mediawiki#specification|BIP 341]]:


witness:  
scriptSig:    (empty)
scriptPubKey: 1 <32_byte_output_key>
   (0x5120{32_byte_output_key})


==Backwards Compatibility==

This BIP is not backwards compatible by design.
An incompatible wallet will not discover these accounts at all and the
user will notice that
something is wrong.

However this BIP uses the same method used in BIPs 44, 49, and 84, so it
should not be difficult
to implement.

==Test vectors==

TBD

==Reference==

* [[bip-0032.mediawiki|BIP32 - Hierarchical Deterministic Wallets]]
* [[bip-0043.mediawiki|BIP43 - Purpose Field for Deterministic Wallets]]
* [[bip-0044.mediawiki|BIP44 - Multi-Account Hierarchy for Deterministic
Wallets]]
* [[bip-0049.mediawiki|BIP49 - Derivation scheme for
P2WPKH-nested-in-P2SH based accounts]]
* [[bip-0084.mediawiki|BIP84 - Derivation scheme for P2WPKH based accounts]]
* [[bip-0341.mediawiki|BIP341 - Taproot: SegWit version 1 spending rules]]


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Taproot Fields for PSBT

[Andrew Chow via bitcoin-dev]

Hi All,

I would like to propose a BIP which defines new fields for Taproot
support in PSBT.

The full text is below, and the rendered file can be found at
https://github.com/achow101/bips/blob/taproot-psbt/bip-taproot-psbt.mediawiki.

Andrew Chow

---


   BIP: taproot-psbt
   Layer: Applications
   Title: Taproot Fields for PSBT
   Author: Andrew Chow 
   Comments-Summary: No comments yet.
   Comments-URI:
https://github.com/bitcoin/bips/wiki/Comments:BIP-taproot-psbt
   Status: draft
   Type: Standards Track
   Created: 2021-06-21
   License: BSD-2-Clause


==Introduction==

===Abstract===

This document proposes additional fields for BIP 174 PSBTv0 and BIP 370
PSBTv2 that allow for
BIP 340/341/342 Taproot data to be included in a PSBT of any version.
These will be fields for
signatures and scripts that are relevant to the creation of Taproot inputs.

===Copyright===

This BIP is licensed under the 2-clause BSD license.

===Motivation===

BIPs 340, 341, and 342 specify Taproot which provides a wholly new way
to create and spend Bitcoin outputs.
The existing PSBT fields are unable to support Taproot due to the new
signature algorithm and the method
by which scripts are embedded inside of a Taproot output. Therefore new
fields must be defined to allow
PSBTs to carry the information necessary for signing Taproot inputs.

==Specification==

The new per-input types are defined as follows:

{|
! Name
! 
! 
!  Description
! 
!  Description
! Versions Requiring Inclusion
! Versions Requiring Exclusion
! Versions Allowing Inclusion
|-
| Taproot Key Spend Signature
| PSBT_IN_TAP_KEY_SIG = 0x13
| None
| No key data '''Why is there no key data for
PSBT_IN_TAP_KEY_SIG'''The signature in a key path spend
corresponds directly with the pubkey provided in the output script. Thus
it is not necessary to provide any metadata that attaches the key path
spend signature to a particular pubkey.
| 
| The 64 or 65 byte Schnorr signature for key path spending a Taproot
output.
|
|
| 0, 2
|-
| Taproot Script Spend Signature
| PSBT_IN_TAP_SCRIPT_SIG = 0x14
|  
| The 32 byte X-only public key concatenated with the 32 byte hash of
the leaf it is part of.
| 
| The 64 or 65 byte Schnorr signature for this pubkey and leaf combination.
|
|
| 0, 2
|-
| Taproot Leaf Script
| PSBT_IN_TAP_LEAF_SCRIPT = 0x15
| 
| The control block for this leaf as specified in BIP 341. The control
block contains the merkle tree path to this leaf.
| 

Re: [bitcoin-dev] Taproot activation proposal "Speedy Trial"

[Andrew Chow via bitcoin-dev]

I don't think we should have a followup deployment start so close to to
timeout of ST. I think it would be better to schedule the followup
around ST, especially since the details around that are fuzzier and
dependent on the results of ST itself.

That being said, I'm not opposed to moving everything one period earlier
or shortening the activation or locked in periods, but I'm not sure if
that is the best course of action right now.

Andrew

On 3/14/21 10:51 PM, Luke Dashjr wrote:
> The last period before timeoutheight here overlaps with the current BIP8(True)
> deployment plan. So if this period specifically were to reach 90% signalling,
> nodes would activate Taproot at height 697536, but ST-only nodes would still
> wait until 709632 instead.
>
> Probably the best solution is to just move this ST window 1 period earlier?
>
> Luke
>
>
> On Saturday 06 March 2021 06:04:22 Andrew Chow via bitcoin-dev wrote:
>> I like this idea.
>>
>> In terms of actual parameters, I propose that we base this Speedy Trial
>> off of BIP 8 with the following parameters:
>> * start height = 681408
>> * timeout height = 695520
>> * lockinontimeout = False
>> * signaling bit = 2
>> * threshold = 1815/2016 blocks (90%)
>>
>> For the extended lockin period, I propose 14112 blocks, which is 7
>> retarget periods. Thus the earliest activation height will be 697536 and
>> the latest activation height will be 709632.
>>
>> This will give us an approximate start time of May 1st 2021 and an
>> approximate timeout time of August 7th 2021, for a total activation
>> period of just over 3 months. The extended lockin period is the same
>> number of blocks as the activation period so that will also be just over
>> 3 months, giving us the latest activation time of November 13th, 2021.
>> If miners activated as soon as possible, the earliest activation time
>> would be August 21st 2021.
>>
>> Additionally, this timeline assumes a mid-April release of Bitcoin Core
>> 0.21.1 containing these parameters. They could be changed to move up if
>> the expected release date were sooner.
>>
>>
>> Andrew Chow
>>
>> On 3/5/21 10:43 PM, David A. Harding via bitcoin-dev wrote:
>>> On the ##taproot-activation IRC channel, Russell O'Connor recently
>>> proposed a modification of the "Let's see what happens" activation
>>> proposal.[1] The idea received significant discussion and seemed
>>> acceptable to several people who could not previously agree on a
>>> proposal (although this doesn't necessarily make it their first
>>> choice).  The following is my attempt at a description.
>>>
>>> 1. Start soon: shortly after the release of software containing this
>>>  proposed activation logic, nodes will begin counting blocks towards
>>>  the 90% threshold required to lock in taproot.[2]
>>>
>>> 2. Stop soon: if the lockin threshold isn't reached within approximately
>>>  three months, the activation attempt fails.  There is no mandatory
>>>  activation and everyone is encouraged to try again using different
>>>  activation parameters.
>>>
>>> 2. Delayed activation: in the happy occasion where the lockin threshold
>>>  is reached, taproot is guaranteed to eventually activate---but not
>>>  until approximately six months after signal tracking started.
>>>
>>> ## Example timeline
>>>
>>> (All dates approximate; see the section below about BIP9 vs BIP8.)
>>>
>>> - T+0: release of one or more full nodes with activation code
>>> - T+14: signal tracking begins
>>> - T+28: earliest possible lock in
>>> - T+104: locked in by this date or need to try a different activation
>>> process - T+194: activation (if lockin occurred)
>>>
>>> ## Analysis
>>>
>>> The goal of Speedy Trial is to allow a taproot activation attempt to
>>> either quickly succeed or quickly fail---without compromising safety in
>>> either case.  Details below:
>>>
>>> ### Mitigating the problems of early success
>>>
>>> New rules added in a soft fork need to be enforced by a large part of
>>> the economy or there's a risk that a long chain of blocks breaking the
>>> rules will be accepted by some users and rejected by others, causing a
>>> chain split that can result in large direct losses to transaction
>>> receivers and potentially even larger indirect losses to holders due to
>>> reduced confidence in the safety of the Bitcoin system.
>>>
>

Re: [bitcoin-dev] Taproot activation proposal "Speedy Trial"

[Andrew Chow via bitcoin-dev]

I like this idea.

In terms of actual parameters, I propose that we base this Speedy Trial
off of BIP 8 with the following parameters:
* start height = 681408
* timeout height = 695520
* lockinontimeout = False
* signaling bit = 2
* threshold = 1815/2016 blocks (90%)

For the extended lockin period, I propose 14112 blocks, which is 7
retarget periods. Thus the earliest activation height will be 697536 and
the latest activation height will be 709632.

This will give us an approximate start time of May 1st 2021 and an
approximate timeout time of August 7th 2021, for a total activation
period of just over 3 months. The extended lockin period is the same
number of blocks as the activation period so that will also be just over
3 months, giving us the latest activation time of November 13th, 2021.
If miners activated as soon as possible, the earliest activation time
would be August 21st 2021.

Additionally, this timeline assumes a mid-April release of Bitcoin Core
0.21.1 containing these parameters. They could be changed to move up if
the expected release date were sooner.


Andrew Chow

On 3/5/21 10:43 PM, David A. Harding via bitcoin-dev wrote:
> On the ##taproot-activation IRC channel, Russell O'Connor recently
> proposed a modification of the "Let's see what happens" activation
> proposal.[1] The idea received significant discussion and seemed
> acceptable to several people who could not previously agree on a
> proposal (although this doesn't necessarily make it their first
> choice).  The following is my attempt at a description.
>
> 1. Start soon: shortly after the release of software containing this
> proposed activation logic, nodes will begin counting blocks towards
> the 90% threshold required to lock in taproot.[2]
>
> 2. Stop soon: if the lockin threshold isn't reached within approximately
> three months, the activation attempt fails.  There is no mandatory
> activation and everyone is encouraged to try again using different
> activation parameters.
>
> 2. Delayed activation: in the happy occasion where the lockin threshold
> is reached, taproot is guaranteed to eventually activate---but not
> until approximately six months after signal tracking started.
>
> ## Example timeline
>
> (All dates approximate; see the section below about BIP9 vs BIP8.)
>
> - T+0: release of one or more full nodes with activation code
> - T+14: signal tracking begins
> - T+28: earliest possible lock in
> - T+104: locked in by this date or need to try a different activation process
> - T+194: activation (if lockin occurred)
>
> ## Analysis
>
> The goal of Speedy Trial is to allow a taproot activation attempt to
> either quickly succeed or quickly fail---without compromising safety in
> either case.  Details below:
>
> ### Mitigating the problems of early success
>
> New rules added in a soft fork need to be enforced by a large part of
> the economy or there's a risk that a long chain of blocks breaking the
> rules will be accepted by some users and rejected by others, causing a
> chain split that can result in large direct losses to transaction
> receivers and potentially even larger indirect losses to holders due to
> reduced confidence in the safety of the Bitcoin system.
>
> One step developers have taken in the past to ensure widespread adoption
> of new consensus rules is programming in a delay between the time software
> with those rules is expected to be released and when the software starts
> tracking which blocks signal for activation.  For example:
>
>  Soft fork| Release| Start  | Delta
>  -+++--
>  BIP68 (v0.12.1)  | 2016-04-15 | 2016-05-11 | 26 days
>  BIP141 (v0.13.1) | 2016-10-27 | 2016-11-18 | 24 days
>
>  Sources: BitcoinCore.org, 
> https://gist.github.com/ajtowns/1c5e3b8bdead01124c04c45f01c817bc
>
> Speedy Trial replaces most of that upfront delay with a backend delay.
> No matter how fast taproot's activation threshold is reached by miners,
> there will be six months between the time signal tracking starts and when
> nodes will begin enforcing taproot's rules.  This gives the userbase even
> more time to upgrade than if we had used the most recently proposed start
> date for a BIP8 activation (~July 23rd).[2]
>
> ### Succeed, or fail fast
>
> The earlier version of this proposal was documented over 200 days ago[3]
> and taproot's underlying code was merged into Bitcoin Core over 140 days
> ago.[4]  If we had started Speedy Trial at the time taproot
> was merged (which is a bit unrealistic), we would've either be less than
> two months away from having taproot or we would have moved on to the
> next activation attempt over a month ago.
>
> Instead, we've debated at length and don't appear to be any closer to
> what I think is a widely acceptable solution than when the mailing list
> began discussing post-segwit activation schemes over a year ago.[5]  I
> think Speedy Trial is a way to generate 

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow via bitcoin-dev]

While working on the reference implementation for this, it occurred to 
me that the Inputs Modifiable flag needs to be more than just a boolean.

If there are existing signatures in the PSBT, then any added inputs 
cannot change the transaction's locktime as all signatures, regardless 
of sighash type, commit to the locktime. Additionally if an input with a 
signature is being added, it also needs to set the locktime for the 
transaction.

It also seems like the SIGHASH_SINGLE bitmap is unnecessary. Signers can 
instead iterate all inputs, check for existing signatures, and extract 
the sighash byte from those signatures to determine whether any are 
SIGHASH_SINGLE. This bitmap doesn't seem to provide much benefit and 
also causes headaches for implementation. So I've decided to remove it.

But it is still useful to know that there are SIGHASH_SINGLE inputs and 
that iteration of the inputs vector will be necessary. It is also useful 
to know that there are already some signatures in the transaction so the 
locktime must be preserved. Thus I would like to change 
PSBT_GLOBAL_TX_MODIFIABLE to include those. I propose making 
PSBT_GLOBAL_TX_MODIFIABLE an 8 bit unsigned little endian integer that 
is treated as a bit field. If bit 0 is set, inputs may be added. This 
will be the Inputs Modifiable flag. If bit 1 is set, outputs may be 
added. This will be the Outputs Modifiable flag. If bit 2 is set, the 
transaction contains signatures and locktime must be preserved. This 
will be the Has Signatures flag. If bit 3 is set, the transaction 
contains SIGHASH_SINGLE inputs and their index pairings must be 
preserved. This will be the Has SIGHASH_SINGLE flag.

Changing PSBT_GLOBAL_TX_MODIFIABLE to a bitfield like this allows us to 
include more conditions that need to be considered when adding inputs 
and outputs. I think these are all of the conditions for now, but with 8 
bits, there is still some space for additional conditions in the future. 
Perhaps it should be changed to be larger if we think there will be more 
conditions, but I think that is unlikely.


Andrew

On 1/15/21 12:28 PM, Andrew Chow wrote:
> Hi All,
>
> I've made some reorganization changes to the way that new PSBT versions
> should be handled in BIP 174 (see
> https://github.com/bitcoin/bips/pull/1055) so PSBTv2 will be submitted
> as a separate BIP. The full document can be read at
> https://github.com/achow101/bips/blob/psbt2/bip-psbt2.mediawiki and I
> have also included it in this email.
>
> I've included Rusty's suggestion for PSBT_GLOBAL_UNDER_CONSTRUCTION and
> made a few modifications. First, the field will be named
> PSBT_GLOBAL_TX_MODIFIABLE and only include the inputs modifiable and
> outputs modifiable flags. The SIGHASH_SINGLE bitmap will be included as
> a separate field PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS. This allows most
> PSBTs to not have to carry around a useless bitmap.
>
> Andrew
>
> ***
>
> 
>     BIP: PSBTv2
>     Layer: Applications
>     Title: PSBT Version 2
>     Author: Andrew Chow 
>     Comments-Summary: No comments yet.
>     Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-PSBT2
>     Status: Draft
>     Type: Standards Track
>     Created: 2021-01-14
>     License: BSD-2-Clause
> 
>
> ==Introduction==
>
> ===Abstract===
>
> This document proposes a second version of the Partially Signed Bitcoin
> Transaction format
> described in BIP 174 which allows for inputs and outputs to be added to
> the PSBT after creation.
>
> ===Copyright===
>
> This BIP is licensed under the 2-clause BSD license.
>
> ===Motivation===
>
> Partially Signed Bitcoin Transaction Version 0 as described in BIP 174
> is unable to have new
> inputs and outputs be added to the transaction. The fixed global
> unsigned transaction
> cannot be changed which prevents any additional inputs or outputs to be
> added.
> PSBT Version 2 is intended to rectify this problem.
>
> An additional benficial side effect is that all information for a given
> input or output will be
> provided by its  or . With
> Version 0, to retrieve
> all of the information for an input or output, data would need to be
> found in two locations:
> the / and the global unsigned
> transaction. PSBT
> Version 2 now moves all related information to one place.
>
> ==Specification==
>
> PSBT Version 2 (PSBTv2) only specifies new fields and field
> inclusion/exclusion requirements.
>
> PSBT_GLOBAL_UNSIGNED_TX must be excluded in PSBTv2.
> PSBT_GLOBAL_VERSION must be included in PSBTv2 and set to
> version number 2'''What happened to version number 1?'''
> Version number 1 is skipped because PSBT Version 0 has been colloquially
> referred to as version 1. Originally this BIP was to be
> version 1, but because it has been colloquially referred to as version 2
> during its design phrase, it was decided to change the
> version number to 2 so that there would not be any confusion.
>
> The new global types for PSBT Version 2 are as follows:
>
> {|
> ! Name
> ! 
> ! 
> !  

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow via bitcoin-dev]

Hi All,

I've made some reorganization changes to the way that new PSBT versions 
should be handled in BIP 174 (see 
https://github.com/bitcoin/bips/pull/1055) so PSBTv2 will be submitted 
as a separate BIP. The full document can be read at 
https://github.com/achow101/bips/blob/psbt2/bip-psbt2.mediawiki and I 
have also included it in this email.

I've included Rusty's suggestion for PSBT_GLOBAL_UNDER_CONSTRUCTION and 
made a few modifications. First, the field will be named 
PSBT_GLOBAL_TX_MODIFIABLE and only include the inputs modifiable and 
outputs modifiable flags. The SIGHASH_SINGLE bitmap will be included as 
a separate field PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS. This allows most 
PSBTs to not have to carry around a useless bitmap.

Andrew

***


   BIP: PSBTv2
   Layer: Applications
   Title: PSBT Version 2
   Author: Andrew Chow 
   Comments-Summary: No comments yet.
   Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-PSBT2
   Status: Draft
   Type: Standards Track
   Created: 2021-01-14
   License: BSD-2-Clause


==Introduction==

===Abstract===

This document proposes a second version of the Partially Signed Bitcoin 
Transaction format
described in BIP 174 which allows for inputs and outputs to be added to 
the PSBT after creation.

===Copyright===

This BIP is licensed under the 2-clause BSD license.

===Motivation===

Partially Signed Bitcoin Transaction Version 0 as described in BIP 174 
is unable to have new
inputs and outputs be added to the transaction. The fixed global 
unsigned transaction
cannot be changed which prevents any additional inputs or outputs to be 
added.
PSBT Version 2 is intended to rectify this problem.

An additional benficial side effect is that all information for a given 
input or output will be
provided by its  or . With 
Version 0, to retrieve
all of the information for an input or output, data would need to be 
found in two locations:
the / and the global unsigned 
transaction. PSBT
Version 2 now moves all related information to one place.

==Specification==

PSBT Version 2 (PSBTv2) only specifies new fields and field 
inclusion/exclusion requirements.

PSBT_GLOBAL_UNSIGNED_TX must be excluded in PSBTv2.
PSBT_GLOBAL_VERSION must be included in PSBTv2 and set to 
version number 2'''What happened to version number 1?'''
Version number 1 is skipped because PSBT Version 0 has been colloquially 
referred to as version 1. Originally this BIP was to be
version 1, but because it has been colloquially referred to as version 2 
during its design phrase, it was decided to change the
version number to 2 so that there would not be any confusion.

The new global types for PSBT Version 2 are as follows:

{|
! Name
! 
! 
!  Description
! 
!  Description
! Versions Requiring Inclusion
! Versions Requiring Exclusion
! Versions Allowing Inclusion
|-
| Transaction Version
| PSBT_GLOBAL_TX_VERSION = 0x02
| None
| No key data
| <32-bit uint>
| The 32-bit little endian signed integer representing the version 
number of the transaction being created. Note that this is not the same 
as the PSBT version number specified by the PSBT_GLOBAL_VERSION field.
| 2
| 0
| 2
|-
| Fallback Locktime
| PSBT_GLOBAL_FALLBACK_LOCKTIME = 0x03
| None
| No key data
| <32-bit uint>
| The 32-bit little endian unsigned integer representing the transaction 
locktime to use if no inputs specify a required locktime.
|
| 0
| 2
|-
| Input Count
| PSBT_GLOBAL_INPUT_COUNT = 0x04
| None
| No key data
| 
| Compact size unsigned integer representing the number of inputs in 
this PSBT.
| 2
| 0
| 2
|-
| Output Count
| PSBT_GLOBAL_OUTPUT_COUNT = 0x05
| None
| No key data
| 
| Compact size unsigned integer representing the number of outputs in 
this PSBT.
| 2
| 0
| 2
|-
| Transaction Modifiable Flags
| PSBT_GLOBAL_TX_MODIFIABLE = 0x06
| None
| No key data
|  
| A single byte boolean (0 for False, 1 for True) representing whether 
inputs can be modified, referred to as the Inputs Modifiable Flag. This 
is followed by a single byte boolean representing whether outputs can be 
modified, referred to as the Outputs Modifiable Flag.
|
| 0
| 2
|-
| SIGHASH_SINGLE Inputs
| PSBT_GLOBAL_SIGHASH_SINGLE_INPUTS = 0x07
| None
| No key data
| 
| A bit vector representing which input indexes use SIGHASH_SINGLE. If 
the bit for an index is set to 1, then the input and output pair at that 
index are tied together with SIGHASH_SINGLE and must be moved together.
|
| 0
| 2
|}

The new per-input types for PSBT Version 2 are defined as follows:

{|
! Name
! 
! 
!  Description
! 
!  Description
! Versions Requiring Inclusion
! Versions Requiring Exclusion
! Versions Allowing Inclusion
|-
| Previous TXID
| PSBT_IN_PREVIOUS_TXID = 0x0e
| None
| No key data
| 
| 32 byte txid of the previous transaction whose output at 
PSBT_IN_OUTPUT_INDEX is being spent.
| 2
| 0
| 2
|-
| Spent Output Index
| PSBT_IN_OUTPUT_INDEX = 0x0f
| None
| No key data
| <32-bit uint>
| 32 bit little endian integer representing the index of the output 
being spent in 

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow via bitcoin-dev]

On 1/7/21 7:40 PM, Rusty Russell wrote:
> Andrew Chow  writes:
>> Hi Rusty,
>>
>> On 1/6/21 6:26 PM, Rusty Russell wrote:
>>> Hi Andrew et al,
>>>
>>>   Very excited to see this progress; thanks for doing all the
>>> work!  Sorry for the delayed feedback, I didn't get to this before the
>>> break.
>>>
 Additionally, I would like to add a new global field:
 * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
  * Key: empty
  * Value: A single byte as a boolean. 0 for False, 1 for True. All
 other values ore prohibited. Must be omitted for PSBTv0, may be omitted
 in PSBTv2.

 PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
 outputs can be added to the PSBT. This flag may be set to True when
 inputs and outputs are being updated, signed, and finalized. However
 care must be taken when there are existing signatures. If this field is
 omitted or set to False, no further inputs and outputs may be added to
 the PSBT.
>>> I wonder if this can be flagged simply by omitting the (AFAICT
>>> redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT?  What
>>> are the purposes of those fields?
>> The purpose of those fields is to know how many input and output maps
>> there are. Without PSBT_GLOBAL_UNSIGNED_TX, there is no way to determine
>> whether a map is an input map or an output map. So the counts are there
>> to allow that.
> Ah, yeah, you need at least the number of input maps :(
>
> It's generally preferable to have sections be self-describing;
> internally if you have a function which takes all the input maps you
> should be able to trivially tell if you're handed the output maps by
> mistake.  Similarly, it would have been nice to have an input map be a
> distinctly marked type from global or output maps.
>
> Nonetheless, that's a bigger change.  You could just require a double-00
> terminator between the global, input and output sections though.
Changing that is a bigger change and I'd rather keep this as minimal as 
possible. Having only new fields is simpler.

>>> For our uses, there would be no signatures at this stage; it's simply a
>>> subdivision of the Creator role.  This role would be terminated by
>>> removing the under-construction marker.  For this, it could be clear
>>> that such an under-construction PSBT SHOULD NOT be signed.
>> There are some protocols where signed inputs are added to transactions.
> Sure, but you can't solve every problem.  We've now created the
> possibility that a PSBT is "under construction" but can't be modified,
> *and* a very invasive requirement to determine that.
>
> I disagree with Andrew's goal here:
>
>>1. PSBT provides no way to modify the set of inputs or outputs after the
>>   Creator role is done.
> It's simpler if, "the under-construction PSBT can be used within the
> Creator role, which can now have sub-roles".
>
> If you really want to allow this (and I think we need to explore
> concrete examples to justify this complexity!), better to add data to
> PSBT_GLOBAL_UNDER_CONSTRUCTION:
> 1. a flag to indicate whether inputs are modifiable.
> 2. a flag to indicate whether outputs are modifiable.
> 3. a bitmap of what inputs are SIGHASH_SINGLE.
>
> If you add a signature which is not SIGHASH_NONE, you clear the "outputs
> modifiable" flag.  If you add a signature which is not
> SIGHASH_ANYONECANPAY, you clear the "inputs modifiable" flag.  If you
> clear both flags, you remove the PSBT_GLOBAL_UNDER_CONSTRUCTION
> altogether.  You similarly set the bitmap depending on whether all sigs
> are SIGHASH_SINGLE.
I think we do need to support adding signed inputs and adding inputs to 
signed transactions. Someone had asked for this feature before. 
Additionally Nicolas Dorier informed me that his NTumbleBit project does 
those things.

In that case, I will include your suggestions for 
PSBT_GLOBAL_UNDER_CONSTRUCTION in the BIP.


Andrew Chow
>
> Cheers,
> Rusty.


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow via bitcoin-dev]

Hi Rusty,

On 1/6/21 6:26 PM, Rusty Russell wrote:
> Hi Andrew et al,
>
>  Very excited to see this progress; thanks for doing all the
> work!  Sorry for the delayed feedback, I didn't get to this before the
> break.
>
>> Additionally, I would like to add a new global field:
>> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>>     * Key: empty
>>     * Value: A single byte as a boolean. 0 for False, 1 for True. All
>> other values ore prohibited. Must be omitted for PSBTv0, may be omitted
>> in PSBTv2.
>>
>> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
>> outputs can be added to the PSBT. This flag may be set to True when
>> inputs and outputs are being updated, signed, and finalized. However
>> care must be taken when there are existing signatures. If this field is
>> omitted or set to False, no further inputs and outputs may be added to
>> the PSBT.
> I wonder if this can be flagged simply by omitting the (AFAICT
> redundant) PSBT_GLOBAL_INPUT_COUNT and PSBT_GLOBAL_OUTPUT_COUNT?  What
> are the purposes of those fields?
The purpose of those fields is to know how many input and output maps 
there are. Without PSBT_GLOBAL_UNSIGNED_TX, there is no way to determine 
whether a map is an input map or an output map. So the counts are there 
to allow that.
> For our uses, there would be no signatures at this stage; it's simply a
> subdivision of the Creator role.  This role would be terminated by
> removing the under-construction marker.  For this, it could be clear
> that such an under-construction PSBT SHOULD NOT be signed.
There are some protocols where signed inputs are added to transactions.
> Otherwise, if an explicit marker is required, I would omit the value and
> simply use its existence to as a flag.  Having two "false" values is
> simply asking for trouble.
Seems reasonable.


Andrew

> Thanks!
> Rusty.
> PS.  Perhaps we should change the name to PBT (Partial Bitcoin
>   Transaction) now, since it's more than just signing...


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow via bitcoin-dev]

Hi All,

The full modified BIP can be read at 
https://github.com/achow101/bips/blob/psbt2/bip-0174.mediawiki.

I will open a PR to the BIPs repo soon after further discussion on this.


Andrew

On 12/22/20 3:12 PM, Andrew Chow wrote:
> Hi All,
>
> I have some updates on this after speaking with some people off-list.
>
> Firstly, the version number will be set to 2. In most discussions, this
> proposal was being referred to as PSBT version 2, so it'll be easier and
> clearer to set the version number to 2.
>
> For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field,
> there will be 2 of them, one for a time based lock time, and the other
> for height based. These will be:
> * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
>     * Key: empty
>     * Value: 32 bit unsigned little endian integer greater than or equal
> to 5 representing the minimum Unix timestamp that this input
> requires to be set as the transaction's lock time. Must be omitted in
> PSBTv0, and may be omitted in PSBTv2
> * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
>     * Key: empty
>     * Value: 32 bit unsigned little endian integer less than 5
> representing the minimum block height that this input requires to be set
> as the transaction's lock time. Must be omitted in PSBTv0, and may be
> omitted in PSBTv2.
>
> Having two lock time fields is necessary due to the behavior where all
> inputs must use the same type of lock time (height or time). Thus if an
> input requires a particular type of lock time, it must set the requisite
> field. Any new inputs being added must be able to accommodate all
> existing inputs' lock time type. This means they either must not have a
> lock time specified (i.e. no OP_CLTV involved), or have branches that
> allow the acceptance of either type. If an input has a lock time type
> that is incompatible with the rest of the transaction, it must not be added.
>
> PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback
> option if no input lock time fields are present. If there are input lock
> times, all lock time calculations must ignore it.
>
> Any role which does lock time calculation will first check if there are
> input lock time fields. If there are not, it must then check for a
> PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the
> transaction's lock time. If it does not, the lock time is 0. If there
> are input lock time fields, it must choose the type which does not
> invalidate any inputs. The lock time is then determined to be the
> maximum value of all of the lock time fields for the chosen type.
>
>
> Additionally, I would like to add a new global field:
> * PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
>     * Key: empty
>     * Value: A single byte as a boolean. 0 for False, 1 for True. All
> other values ore prohibited. Must be omitted for PSBTv0, may be omitted
> in PSBTv2.
>
> PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and
> outputs can be added to the PSBT. This flag may be set to True when
> inputs and outputs are being updated, signed, and finalized. However
> care must be taken when there are existing signatures. If this field is
> omitted or set to False, no further inputs and outputs may be added to
> the PSBT.
>
> Several rules must be followed to ensure that adding additional inputs
> and outputs will not invalidate existing signatures. First, an input or
> output adder must check for any existing signatures in all of the other
> inputs. If there are none, the input or output may be added in any
> position. If there are one or more signatures, each signature's sighash
> type must be examined. Inputs may only be added if all existing
> signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all
> existing signatures use SIGHASH_NONE. If an input has a signature using
> SIGHASH_SINGLE, the same number of inputs and outputs must be added
> before that input and it's corresponding output. For all other sighash
> types (i.e. SIGHASH_ALL and any future sighash types), no inputs or
> outputs may be added to the PSBT. Specific exceptions can be made in the
> future for additional sighash types.
>
> Furthermore, these newly added inputs must follow additional lock time
> rules. Because all signatures, regardless of sighash type, sign the
> transaction lock time, newly added inputs when there are existing
> signatures must have the same type of lock time used in the transaction,
> and must be less than or equal to the transaction lock time. It must not
> cause the transaction lock time to change, otherwise the signatures will
> be invalidated.
>
>
> Lastly, to uniquely identify transactions for combiners, a txid can be
> computed from the information present in the PSBT. Internally, combiners
> can create an unsigned transaction given the transaction version, the
> input prevouts, the outputs, and the computed locktime. This can then be
> used to calculate a txid and thus used as a way to identify PSBTs.
> Combiners will 

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow via bitcoin-dev]

Hi,

On 12/22/20 10:30 PM, fiatjaf wrote:
> Hi Andrew.
>
> I'm just a lurker here and I have not much experience with PSBTs, but still 
> let me pose this very obvious question and concern: isn't this change going 
> to create a compatibility nightmare, with some software supporting version 1, 
> others supporting version 2, and the ones that care enough about UX and are 
> still maintained being forced to support both versions -- and for no very 
> important reason except some improvements in the way data is structured?
No, it is not just "improvements in the way data is structured."

The primary reason for these changes is to allow PSBT to properly 
support adding inputs and outputs. This is a feature that many people 
have requested, and the ways that people have been doing it are honestly 
just hacks and not really the right way to be doing that. These changes 
allow for that feature to be supported well.

Furthermore, it is possible to downgrade and upgrade PSBTs between the 
two versions, once all inputs and outputs have been decided. Since 
PSBTv2 is essentially just taking all of the normal transaction fields 
and grouping them all with the rest of the data for those inputs and 
outputs, it is easy to reconstruct a global unsigned transaction and 
turn a PSBTv2 into a PSBTv0. It is likewise just as easy to go the other 
way and break apart the global unsigned tx to turn a PSBTv0 into a 
PSBTv2. Originally, I had considered requiring that once a transaction 
was fully constructed it must be downgraded to a PSBTv0, but the 
structure changes that were made do make it easier to work with PSBT so 
I decided not to add this requirement.

Perhaps to maintain compatibility PSBT_GLOBAL_UNSIGNED_TX shouldn't be 
disallowed in PSBTv2 once the transaction is constructed? It would make 
things much more confusing though as it would no longer be a clean break.


Andrew Chow

> Ultimately I don't think it should matter if some data is structured in 
> not-the-best-possible way, as long as it is clear enough for the computer and 
> for the libraries already written to deal with it. Backwards-compatibility 
> and general interoperability is worth much more than anything else in these 
> cases.
>
> Also let me leave this article here, which I find very important (even if for 
> some reason it ends up not being relevant to this specific case): 
> http://scripting.com/2017/05/09/rulesForStandardsmakers.html
>
>    On Tue, 22 Dec 2020 17:12:22 -0300 Andrew Chow via bitcoin-dev 
>  wrote 
>   > Hi All,
>   >
>   > I have some updates on this after speaking with some people off-list.
>   >
>   > Firstly, the version number will be set to 2. In most discussions, this
>   > proposal was being referred to as PSBT version 2, so it'll be easier and
>   > clearer to set the version number to 2.
>   >
>   > For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field,
>   > there will be 2 of them, one for a time based lock time, and the other
>   > for height based. These will be:
>   > * PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
>   >* Key: empty
>   >* Value: 32 bit unsigned little endian integer greater than or equal
>   > to 5 representing the minimum Unix timestamp that this input
>   > requires to be set as the transaction's lock time. Must be omitted in
>   > PSBTv0, and may be omitted in PSBTv2
>   > * PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
>   >* Key: empty
>   >* Value: 32 bit unsigned little endian integer less than 5
>   > representing the minimum block height that this input requires to be set
>   > as the transaction's lock time. Must be omitted in PSBTv0, and may be
>   > omitted in PSBTv2.
>   >
>   > Having two lock time fields is necessary due to the behavior where all
>   > inputs must use the same type of lock time (height or time). Thus if an
>   > input requires a particular type of lock time, it must set the requisite
>   > field. Any new inputs being added must be able to accommodate all
>   > existing inputs' lock time type. This means they either must not have a
>   > lock time specified (i.e. no OP_CLTV involved), or have branches that
>   > allow the acceptance of either type. If an input has a lock time type
>   > that is incompatible with the rest of the transaction, it must not be 
> added.
>   >
>   > PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback
>   > option if no input lock time fields are present. If there are input lock
>   > times, all lock time calculations must ignore it.
>   >
>   > Any role which does lock time calculation will first check if there are
>   > input lock time fields. If there are not, it mu

Re: [bitcoin-dev] New PSBT version proposal

[Andrew Chow via bitcoin-dev]

Hi All,

I have some updates on this after speaking with some people off-list.

Firstly, the version number will be set to 2. In most discussions, this 
proposal was being referred to as PSBT version 2, so it'll be easier and 
clearer to set the version number to 2.

For lock times, instead of a single  PSBT_IN_REQUIRED_LOCKTIME field, 
there will be 2 of them, one for a time based lock time, and the other 
for height based. These will be:
* PSBT_IN_REQUIRED_TIME_LOCKTIME = 0x10
   * Key: empty
   * Value: 32 bit unsigned little endian integer greater than or equal 
to 5 representing the minimum Unix timestamp that this input 
requires to be set as the transaction's lock time. Must be omitted in 
PSBTv0, and may be omitted in PSBTv2
* PSBT_IN_REQUIRED_HEIGHT_LOCKTIME = 0x11
   * Key: empty
   * Value: 32 bit unsigned little endian integer less than 5 
representing the minimum block height that this input requires to be set 
as the transaction's lock time. Must be omitted in PSBTv0, and may be 
omitted in PSBTv2.

Having two lock time fields is necessary due to the behavior where all 
inputs must use the same type of lock time (height or time). Thus if an 
input requires a particular type of lock time, it must set the requisite 
field. Any new inputs being added must be able to accommodate all 
existing inputs' lock time type. This means they either must not have a 
lock time specified (i.e. no OP_CLTV involved), or have branches that 
allow the acceptance of either type. If an input has a lock time type 
that is incompatible with the rest of the transaction, it must not be added.

PSBT_GLOBAL_PREFERRED_LOCKTIME is changed to purely be the fallback 
option if no input lock time fields are present. If there are input lock 
times, all lock time calculations must ignore it.

Any role which does lock time calculation will first check if there are 
input lock time fields. If there are not, it must then check for a 
PSBT_GLOBAL_PREFERRED_LOCKTIME. If this field exists, its value is the 
transaction's lock time. If it does not, the lock time is 0. If there 
are input lock time fields, it must choose the type which does not 
invalidate any inputs. The lock time is then determined to be the 
maximum value of all of the lock time fields for the chosen type.


Additionally, I would like to add a new global field:
* PSBT_GLOBAL_UNDER_CONSTRUCTION = 0x05
   * Key: empty
   * Value: A single byte as a boolean. 0 for False, 1 for True. All 
other values ore prohibited. Must be omitted for PSBTv0, may be omitted 
in PSBTv2.

PSBT_GLOBAL_UNDER_CONSTRUCTION is used to signal whether inputs and 
outputs can be added to the PSBT. This flag may be set to True when 
inputs and outputs are being updated, signed, and finalized. However 
care must be taken when there are existing signatures. If this field is 
omitted or set to False, no further inputs and outputs may be added to 
the PSBT.

Several rules must be followed to ensure that adding additional inputs 
and outputs will not invalidate existing signatures. First, an input or 
output adder must check for any existing signatures in all of the other 
inputs. If there are none, the input or output may be added in any 
position. If there are one or more signatures, each signature's sighash 
type must be examined. Inputs may only be added if all existing 
signatures use SIGHASH_ANYONECANPAY. Outputs may only be added if all 
existing signatures use SIGHASH_NONE. If an input has a signature using 
SIGHASH_SINGLE, the same number of inputs and outputs must be added 
before that input and it's corresponding output. For all other sighash 
types (i.e. SIGHASH_ALL and any future sighash types), no inputs or 
outputs may be added to the PSBT. Specific exceptions can be made in the 
future for additional sighash types.

Furthermore, these newly added inputs must follow additional lock time 
rules. Because all signatures, regardless of sighash type, sign the 
transaction lock time, newly added inputs when there are existing 
signatures must have the same type of lock time used in the transaction, 
and must be less than or equal to the transaction lock time. It must not 
cause the transaction lock time to change, otherwise the signatures will 
be invalidated.


Lastly, to uniquely identify transactions for combiners, a txid can be 
computed from the information present in the PSBT. Internally, combiners 
can create an unsigned transaction given the transaction version, the 
input prevouts, the outputs, and the computed locktime. This can then be 
used to calculate a txid and thus used as a way to identify PSBTs. 
Combiners will need to do this for all version 2 PSBTs in order to avoid 
combining distinct transactions.


Andrew Chow

On 12/9/20 5:25 PM, Andrew Chow wrote:
> Hi All,
>
> I would like to propose a new PSBT version that addresses a few
> deficiencies in the current PSBT v0. As this will be backwards
> incompatible, a new PSBT version will be used, v1.
>
> The 

[bitcoin-dev] New PSBT version proposal

[Andrew Chow via bitcoin-dev]

Hi All,

I would like to propose a new PSBT version that addresses a few 
deficiencies in the current PSBT v0. As this will be backwards 
incompatible, a new PSBT version will be used, v1.

The primary change is to truly have all input and output data for each 
in their respective maps. Instead of having to parse an unsigned 
transaction and lookup some data from there, and other data from the 
correct map, all of the data for an input will be contained in its map. 
Doing so also disallows PSBT_GLOBAL_UNSIGNED_TX in this new version. 
Thus I propose that the following fields be added:

Global:
* PSBT_GLOBAL_TX_VERSION = 0x02
   * Key: empty
   * Value: 32-bit little endian unsigned integer for the transaction 
version number. Must be provided in PSBT v1 and omitted in v0.
* PSBT_GLOBAL_PREFERRED_LOCKTIME = 0x03
   * Key: empty
   * Value: 32 bit little endian unsigned integer for the preferred 
transaction lock time. Must be omitted in PSBT v0. May be provided in 
PSBT v1, assumed to be 0 if not provided.
* PSBT_GLOBAL_INPUT_COUNT = 0x04
   * Key: empty
   * Value: Compact size unsigned integer. Number of inputs in this 
PSBT. Must be provided in PSBT v1 and omitted in v0.
* PSBT_GLOBAL_OUTPUT_COUNT = 0x05
   * Key: empty
   * Value: Compact size unsigned integer. Number of outputs in this 
PSBT. Must be provided in PSBT v1 and omitted in v0.

Input:
* PSBT_IN_PREVIOUS_TXID = 0x0e
   * Key: empty
   * Value: 32 byte txid of the previous transaction whose output at 
PSBT_IN_OUTPUT_INDEX is being spent. Must be provided in PSBT v1 and 
omitted in v0.
* PSBT_IN_OUTPUT_INDEX = 0x0f
   * Key: empty
   * Value: 32 bit little endian integer for the index of the output 
being spent. Must be provided in PSBT v1 and omitted in v0.
* PSBT_IN_SEQUENCE = 0x0f
   * Key: empty
   * Value: 32 bit unsigned little endian integer for the sequence 
number. Must be omitted in PSBT v0. May be provided in PSBT v1 assumed 
to be max sequence (0x) if not provided.
* PSBT_IN_REQUIRED_LOCKTIME = 0x10
   * Key: empty
   * Value: 32 bit unsigned little endian integer for the lock time that 
this input requires. Must be omitted in PSBT v0. May be provided in PSBT 
v1, assumed to be 0 if not provided.

Output:
* PSBT_OUT_VALUE = 0x03
   * Key: empty
   * Value: 64-bit unsigned little endian integer for the output's 
amount in satoshis. Must be provided in PSBT v1 and omitted in v0.
* PSBT_OUT_OUTPUT_SCRIPT = 0x04
   * Key: empty
   * Value: The script for this output. Otherwise known as the 
scriptPubKey. Must be provided in PSBT v1 and omitted in v0.

This change allows for PSBT to be used in the construction of 
transactions. With these new fields, inputs and outputs can be added as 
needed. One caveat is that there is no longer a unique transaction 
identifier so more care must be taken when combining PSBTs. 
Additionally, adding new inputs and outputs must be done such that 
signatures are not invalidated. This may be harder to specify.

An important thing to note in this proposal are the fields 
PSBT_GLOBAL_PREFERRED_LOCKTIME and PSBT_IN_REQUIRED_LOCKTIME. A Bitcoin 
transaction only has a single locktime yet a PSBT may have multiple 
locktimes. To choose the locktime for the transaction, finalizers must 
choose the maximum of all of the *_LOCKTIME fields. 
PSBT_IN_REQUIRED_LOCKTIME is added because some inputs, such as those 
involving OP_CHECKLOCKTIMEVERIFY, require a specific minimum locktime to 
be set. This field allows finalizers to choose a locktime that is high 
enough for all inputs without needing to understand the scripts 
involved. The PSBT_GLOBAL_PREFERRED_LOCKTIME is the locktime to use if 
no inputs require a particular locktime.

As these changes disallow the PSBT_GLOBAL_UNSIGNED_TX field, PSBT v1 
needs the version number bump to enforce backwards incompatibility. 
However once the inputs and outputs of a PSBT are decided, a PSBT could 
be "downgraded" back to v0 by creating the unsigned transaction from the 
above fields, and then dropping these new fields.

If the list finds that these changes are reasonable, I will write a PR 
to modify BIP 174 to incorporate them.

Thanks,
Andrew Chow

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] PSBT Addition (BIP 174) for authenticating source/output PSBT files

[Andrew Chow via bitcoin-dev]

On 1/13/20 3:29 PM, Peter D. Gray wrote:
> The Signer may be signing a PSBT that was corrupted by the MitM,
> but at least later users of the signed PSBT can detect that occured.
> At present, they do not know what the input PSBT content was when
> it got to the Signer.

But the MiTM on the way to the other roles will still see that signed
PSBT, and if the Signer has produced a signature such that someone can
get the private key, that MiTM will have the private key before the
transaction is broadcast. If this isn't part of your threat model, I
think it should be; I don't think it is reasonable to say that you are
only protecting against MiTM for one specific leg of the entire protocol.

> If we use a fixed-width signature, such as just R+S bytes (64 bytes),
> and not DER-encoding, then the signature is a fixed distance from
> the last byte of the file. A conservative PSBT parser could start
> by verifying the signature exists and is valid, before parsing the
> rest of the file. (It would need to use the pubkeys from the original
> PSBT, which it would ideally have on-hand already to verify the source
> PSBT to the Coldcard.)

Why the end? This brings up a particular implementation detail I didn't
want to get into since I was opposing the idea conceptually, but I don't
think that 2 new types are necessary. We absolutely do not need nor
should we have any global data (and the auth sig is absolutely global
data) in input or output specific fields. The outputs really should be
independent of the other inputs and outputs. So having the last output
have the signature is a layer violation.

Why put it at the end? If you want a byte offset, just put the signature
in the globals as the first kv pair.

> I agree that Finalizers cannot access the Bitcoin private keys, but
> they still have stacks that can overflow, buffers that can be overrun
> and so on. Perhaps if sighash is not SIGHASH_ALL, there are dangerous
> things they can be tricked into... I don't know, but at least we
> should make it possible to detect these cases. My goal is detection.

But that shouldn't matter to the Finalizer. It isn't the Finalizer's job
to enforce that the Signers followed a specific signing policy. If the
Signer chose to sign with a "dangerous policy", that's up to the signer
and the Finalizers shouldn't have anything to do with that.

> No, I am not proposing anyone re-construct PSBT's... My proposal
> is really only helpful if you have the full original PSBT on hand
> (or its digest). For ultimate safety I would recommend checking the
> incoming PSBT's signature is valid before parsing it.(If the
> signature is fixed-length, see above.)

That's another thing I don't understand about your proposal. Your
signature covers the "Original PSBT" which is really nebulous and could
anything. This doesn't make sense to me. Everyone has to somehow get the
same "Original PSBT" so you are assuming there's no MiTM in that initial
distribution (seems like an oversight in your threat model).

But then your "Original PSBT" can also be in a number of different
states, and your signature wouldn't cover some things. For example, the
"Original" could have just some of the UTXOs and some of the scripts,
not everything. So in later steps of the process, the MiTM protection
doesn't cover those things, so an attacker could modify them with no
effect on the signature.

> In the USB protocol between Coldcard and desktop, we do end-to-end
> encryption with a session key picked via diff-hel so we're doing
> our best there against MitM. However, our customers love the air-gap
> feature which involves lots of sneakernet handling of MicroSD cards.
> I don't want to force them into handling paired files, like detacted
> signatures, and I was hoping this would be a good way to move the
> signatures inside the PSBT files already being moved about.

You could put them in an archive (tarfile) so it's still just one file
being copied from the SD card. You already have archive creation on the
coldcard for backup creation anyways.

***

I guess what I don't get about this proposal is your threat model and
what specifically you are protecting against. It seems like this is only
protecting against the specific leg from a specific combined
Updater/Finalizer to and from its respective Signer. But this is not
always the use case and this isn't very generic. Other places that there
could be MiTM aren't covered.

I also don't get what a MiTM could even do. If your parser is vulnerable
to the standard programming vulns (buffer overflows, stack overflows,
etc.), ISTM you will still run into those with just a normal PSBT. If
you don't, then a MiTM can't trigger one there. And likewise for
signature issues; if your signer might produce a private key leaking
signature, then it will probably do that with a non-MiTM'd PSBT, and if
not, MiTM isn't going to change that.

Andrew

> 
> ---
> Peter D. Gray  ||  Founder, Coinkite  ||  Twitter: @dochex  ||  GPG: A3A31BAD 
> 5A2A5B10
> 
> 

Re: [bitcoin-dev] PSBT Addition (BIP 174) for authenticating source/output PSBT files

[Andrew Chow via bitcoin-dev]

On 1/13/20 9:28 AM, Peter D. Gray wrote:
> I don't have a specific attack in mind, but these signatures, if
> adopted by the community at large, will allow detection of-, and
> could mitigate damage from-, some broad "bug-classes".
> 
> Consider if the PSBT Signer (hardware wallet) has bugs. Perhaps if
> you tweak the PSBT in some unnatural way it produces output that
> reveals the private key (duplicate k-value perhaps), or corrupts
> the display of the transaction in helpful (to the attacker) ways
> (typically case: output hidden as change).

Since the PSBT is to be signed by one of the Signers for the PSBT, I
don't see how this is useful. If it is mutated and the signer has bugs,
especially parsing bugs, the Signer also adding its signature won't
help. In your proposal, it is the Signer who adds the signature, so it
will receive a PSBT without auth sigs and thus that could be mutated to
trigger those bugs anyways.

> There could also be bugs in the Combiner/Finalizer which the MiTM
> wants to trigger. Legimate files, signed by the PSBT Signer, will not
> contain those attacks, so are "safer" to process, even if your
> Combiner's PSBT parser has bugs or is tragically dumb.

The job of Combiners is fairly limited and is really just related to
parsing the PSBT into some internal object then shuffling those fields
around. In that case, any bugs an attacker would want to exploit have to
be deserialization bugs, in which case, your auth sigs don't help. The
Combiner still has to deserialize the PSBT to get the signature, then it
needs to re-serialize the PSBT to check that signature. An attacker
could insert bad bytes into the PSBT which causes problems during
deserialization, before the Combiner is able to check the signature.

For Finalizers, since its job is to construct the final
scriptSig/scriptWitness, at worst, all it can do is produce an invalid
transaction. Finalizers don't have access to the private keys so there's
no bug possible that can result in a Finalizer producing a transaction
that reveals the private key.

> 
> That's just it, when we receive a signed PSBT, at present we don't
> know *what* was signed without a complete understanding of the
> transaction, the input UTXO (at least syntactially), and PSBT file
> contents.  If there are bugs in that understanding (ie. checks we
> all know are needed, but no-one actually implemented) then we might
> transmit an harmful transaction, or continue to process a file
> that has been corrupted-with-intent by a MiTM.

ISTM the same is true of your proposal. You need to deserialize the PSBT
and then figure out which fields were "original" and in what order. If
there is a bug in your deserialization, an attacker can still exploit
that. And if there is a bug in your reconstruction of "original", you'll
have false positives.

> It's fine to say that, but in an embedded environment, with very
> limited memory like the Coldcard, PGP isn't an option (signing vs.
> signature verification). I want to leverage the existing crypto and
> PKI that we already have in play.

My point was that you can achieve your MiTM protection by having the
signature separate from the PSBT. You can still make your ECDSA
signature and send it along with the PSBT, and you can do it with fixed
or exchanged keys, no need for parsing the PSBT itself. It can be part
of the transport protocol, not part of the data that is being transferred.

Andrew

> 
>> On 1/11/20 3:17 PM, Dmitry Petukhov via bitcoin-dev wrote:
> ... [many valid points, repeated by Andrew] ...
>>> If there is MitM, checking something at Finalizer is likely too
>>> late - the party that can intercept PSBTs can finalize before the
>>> legitimate Finalizer and broadcast the transaction.
> 
> Yes, that is a problem which is proposal does not address. If the
> MitM has control over both directions, in and out, then whatever
> he or she was trying to do will still happen. Personally, I'm okay
> with that as a limition, but using the same signatures features,
> and a pre-shared public key between the PSBT Creator and the Signer,
> we could block the Signer from looking at MitM'ed files. (The Signer
> would require and verify incoming unsigned PSBT to contain the
> last-output-section-signature thing.) I'm not planning on supporting
> that on the Coldcard (at least not yet), but with the proposed
> additions, it is possible to do without further changes to the PSBT
> spec.
> 
>>> Participants can work from the same PSBT ...
>>> either pass two files (original and updated), or work out which fields
>>> (key-value blobs) to remove to get the 'source' PSBT (which might not be
>>> trivial with presense of proprietary and unknown fields). Even if you
>>> know which key-value pairs to remove, there is no requirement for
>>> ordering of the fields, and some signer can serialize them in different
>>> order after dserialize/sign/add-signatures/re-serialize operation.
> ...
>>> Introducing additional ordering or other structure 

Re: [bitcoin-dev] PSBT Addition (BIP 174) for authenticating source/output PSBT files

[Andrew Chow via bitcoin-dev]

I agree with Dimitry. I don't see the point of having the MiTM
protection within the PSBT structure itself, in addition to the fact
that adding new fields is largely unnecessary. In fact, I'm not quite
sure what kind of attack you are trying to defend against with this
proposal.

If there is a MiTM who can modify your PSBT, then they can just modify
the result the signed PSBT to drop the auth signatures. Furthermore, any
modifications to scripts or UTXOs would just result in an invalid
signature, so only time is wasted. But you'll just waste time anyways
when you see a failed auth sig.

Additionally, when a signer processes a PSBT, it will either accept the
PSBT and add a signature for its inputs, or reject it and do nothing.
Given this behavior (and I assume you aren't going to add auth sigs for
rejected PSBTs because that doesn't make any sense), then you already
have a signature there that covers everything your auth signature would
cover. So just verify those signatures instead; for any inputs with
signatures, everything you need to verify them are already there.

Lastly, IMO, if you want MiTM protection, then you should do your
protection with out of band communication. Just PGP sign the PSBT (or
something similar) and send the signature along separately.

Andrew

On 1/11/20 3:17 PM, Dmitry Petukhov via bitcoin-dev wrote:
> 
> I am not sure that this particular task should be done with data
> embedded in PSBT itself, and not with some sort of container that
> includes PSBT and the authentication information.
> 
> The benefit seems to be in reusing PSBT structure for compatibilty, and
> this might be a valid way, although I do not agree with some of your
> points. I elaborate below:
> 
>> 1) In the PSBT globals section, a signature over the "source" PSBT
>> file. It would cover all the bytes of the original PSBT file, as
>> it was received by the Signer.
> 
> The problem of authenticating the contents of PSBT is independent of
> the signing action. PSBT might be altered on the path from Creator to
> Signer. Therefore you cannot always say that Signer will be an
> authority over 'correctness' of PSBT.
> 
>> - At the end of the signing process, the Finalizer should check all
>> the Signers have worked from the same PSBT file (assuming that's
>> the flow expected)
> 
> If there is MitM, checking something at Finalizer is likely too
> late - the party that can intercept PSBTs can finalize before the
> legitimate Finalizer and broadcast the transaction.
> 
> Participants can work from the same PSBT file if they all receive the
> same PSBT, and not working in chain where next particpant receives
> updated PSBT from the previous participant. Otherwise they will need to
> either pass two files (original and updated), or work out which fields
> (key-value blobs) to remove to get the 'source' PSBT (which might not be
> trivial with presense of proprietary and unknown fields). Even if you
> know which key-value pairs to remove, there is no requirement for
> ordering of the fields, and some signer can serialize them in different
> order after dserialize/sign/add-signatures/re-serialize operation.
> 
> Introducing additional ordering or other structure requirements over
> simple key-value structure will add complexity to PSBT processing, and
> adding complexity on such a basic level should have really serious
> reasons, because that increases effort required for even basic
> implementations and increases chance of bugs.
> 
> If there is some authority on the 'correctness' of 'original' PSBT
> (all particpants receive same PSBT at the start), particpants should
> check the signature by that authority. That authority might use
> the key used only for authentication, and not in the tx signing.
> 
> If particpants send PSBT in chain after adding their signatures, then
> each participant can add their signature to say 'the contents
> of PSBT after my updates should match this hash'.
> 
> The signatures of previous participants in the chain most likely do not
> matter because of difficulty of restoring the contents of PSBT as it
> was before the previous particpant, if you do not pass _all_ the PSBTs
> (which is excessive).
> 
>> 2) In the output section, specifically, the last key/value pair of
>> the last output of the transaction, I want to add a similar signature,
>> again signed by one of the keys used in the signing process. This
>> signature will cover all the bytes of the resulting (signed) PSBT
>> up to that point. Because it is the last output of the output
>> section, that signature will be the last few bytes of the PSBT file.
>> By "appending" the signature in this way, it's easier to validate
>> and create the signature, without blanking the signature area during
>> digest step.
> 
> This will introduce unnecessary higher-level structure to PSBT for the
> reasons that I do not find strong enough for the amount of complexity
> added.
> 
> Also, as I said above, you likely do not need more than one
> 

Re: [bitcoin-dev] Base64-encoded descriptors

[Andrew Chow via bitcoin-dev]

Hi All,

Just a few comments about choosing an encoding and why this is even
being proposed.


On Wednesday, December 25, 2019 12:17 PM, William Casarin via
bitcoin-dev  wrote:

> I don't think encoding descriptors is a good idea. Encoding makes more
> sense if it's non-human-readable binary data that you want transfer
> over
> a plaintext channel.
>
> Descriptors aren't binary data, and have a wealth of useful
> information
> that you can view at a glance. Obfuscating this information just to
> gain
> the ability to copy-paste doesn't seem like a good idea.

The main reasons this was proposed in the first place is because of
concerns that users will be unwilling to use or be confused by descriptors.
There is a concern that users will not understand the commas,
parentheses, brackets, etc. syntax of descriptors and thus only copy
part of it.
There is also the concern that users will see this code-like syntax and
be intimidated by it so they will not want to handle them.

So my (offhanded) suggestion was to encode it in some way to just make
it look like some magic string that they need to handle as one unit.


> so I'm a bit sad that base64 was
> chosen. base64 isn't really good for double-click copy-pasting, it
> contains characters such as +/= which aren't always included when
> double-clicking. I prefer bech32, base58 or base62.

On Tuesday, December 24, 2019 2:09 PM, Spencer Dupre` via bitcoin-dev
 wrote:

> Sounds like a good UX improvement, but do we really need to introduce
a new encoding? Perhaps bech32 could be used instead.

On Tuesday, December 24, 2019 2:25 PM, Pavol Rusnak via bitcoin-dev
 wrote:

> I'd rather see something using Base58 or even better Bech32. Base64 is
not URL/QR code friendly.

A different encoding scheme could certainly be used. Base64 was
suggested in my comments to Chris and others as it is a well known
encoding scheme that doesn't already define its own checksum as Base58
and Bech32 do. This is an important detail because descriptors *also*
have their own checksum scheme.

While other encoding methods could be used, I do want to point out that
it would be nice to stick to things that already exist. We could use a
bech32-like encoding, just with the different BCH code that descriptors
use instead of the bech32 code, but calling that bech32 would be a bit
confusing. And I don't think we should use Base58 at all.

On Tuesday, December 24, 2019 8:02 PM, Trey Del Bonis via bitcoin-dev
 wrote:

> Part of the aversion to using bech32 may be that the BCH code used in
> bech32 for error detection doesn't hold up for messages longer than
> some length (that I can't remember off the top of my head).  It still
> encodes and decodes perfectly well but a decoder won't be guaranteed
> to detect potential errors, so that's somewhat wasted there.  Maybe
> someone should define a derivatives of bech32 that retains error
> detection properties for longer message lengths, such as those used in
> lightning invoices.

Descriptors already have their own BCH code for descriptor checksums
optimized for their length and character rset. This can be repurposed to
be used with whatever encoding scheme is chosen so long as the
encoding's character set is covered by the descriptor checksum character
set. The checksum's character set is fairly large and covers all(?)
characters on a standard keyboard so that descriptors could be expanded
with other features in the future. Thus it should cover any encoding
scheme that is suggested.

More information about the descriptor checksum can be found at
https://github.com/bitcoin/bitcoin/blob/master/src/script/descriptor.cpp#L26

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] PSBT_GLOBAL_XPUB: Version bytes in xpub

[Andrew Chow via bitcoin-dev]

The rationale was that xpubs was already a predefined standard which many 
software already have serialization code for. It's simpler to just reuse what 
has been defined before.

IMO, the version bytes don't matter and should be ignored. In the proposed 
implementation to Bitcoin Core, the version bytes are ignored.

Andrew

On 11/15/19 5:29 PM, Ian Kroozbi via bitcoin-dev wrote:

> What’s the rational for including the version bytes in the xpub in the 
> GLOBAL_XPUB field? This was briefly touched upon in an earlier email from 
> Stepan, but I don’t think it was every fully addressed.
>
>> I am not sure if we need the whole xpub or keeping chain_code and
>> public_key is enough, but I would suggest to keep other data
>> just in case. For example, keeping prefix that defines if the key is used
>> for testnet or mainnet may be useful.
>
> The version bytes seem to be superfluous data considering the transaction 
> format and the rest of the PSBT key-values are network agnostic. If we wanted 
> to attach network data to the PSBT then I think that would be better served 
> by using a new key.
>
> There is also a potential issue where we could have conflicting versions on 
> different xpubs in the PSBT. If we remove the version bytes then we can 
> eliminate this possibility.
>
> I haven’t formed an opinion on whether or not the other data outside of the 
> public key and chain code should be included, but I think it would be good to 
> discuss any rational for either including it or omitting it.___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility

[Andrew Chow via bitcoin-dev]

I spoke to some people OOB and they said that they didn't really like
the idea of having a prefix string (partially because they've already
implemented some proprietary types by simply squatting on unused types).
Matching the prefix string adds additional complexity to the parser
code. The main concern is that people won't want to actually follow the
spec for proprietary types and instead just use some unused type value.
So I think instead we should do:

{0xFC}|{private_type}

and the prefix string can be optional (and strongly recommended) after that.

Since public parsers won't really be enforcing the rules for proprietary
types, I don't think it really makes sense to specify and enforce how
they should be. After all, the key is really just an opaque data blob.

In the same vein, it would probably be useful to allow multiple types
for proprietary use as originally proposed to make implementation of
these easier. If more type are needed, then the private type
construction can be used.


Andrew

On 8/1/19 1:57 PM, Andrew Chow wrote:
> 
> It seems like the consensus is that we should use Compact Size Unsigned
> Integers for the field types, so we will do that. The types will be
> minimally encoded CSUints.
> 
> For the proprietary types, I will use Dimitry's and Andrew Poelstra's
> (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-March/016713.html)
> suggestion. There will be one proprietary type, 0xFC. This will be
> followed by a variable length string that is a vendor specific prefix
> that serves as a unique identifier to help with preventing usage of
> proprietary types outside of private contexts. This will then be
> followed by the actual type for the data, as defined by the user of the
> proprietary type.
> 
> The prefix will just be a string, prefixed with a compact size unsigned
> integer. If no prefix is wanted, then a single 0x00 byte can be used.
> 
> For public software, there is no need to handle these proprietary types
> at all so they do not need to check the string or the data type. It is
> not necessary to enforce the above rules about proprietary types except
> that they use the proprietary type value.
> 
> 
> Andrew Chow
> 

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility

[Andrew Chow via bitcoin-dev]

It seems like the consensus is that we should use Compact Size Unsigned
Integers for the field types, so we will do that. The types will be
minimally encoded CSUints.

For the proprietary types, I will use Dimitry's and Andrew Poelstra's
(https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-March/016713.html)
suggestion. There will be one proprietary type, 0xFC. This will be
followed by a variable length string that is a vendor specific prefix
that serves as a unique identifier to help with preventing usage of
proprietary types outside of private contexts. This will then be
followed by the actual type for the data, as defined by the user of the
proprietary type.

The prefix will just be a string, prefixed with a compact size unsigned
integer. If no prefix is wanted, then a single 0x00 byte can be used.

For public software, there is no need to handle these proprietary types
at all so they do not need to check the string or the data type. It is
not necessary to enforce the above rules about proprietary types except
that they use the proprietary type value.


Andrew Chow

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility

[Andrew Chow via bitcoin-dev]

Hi,

On 7/31/19 12:19 PM, Dmitry Petukhov wrote:
> 
> I think private formats should have at least a basic format: they
> should start with a prefix. This way different prviate formats can be
> distinguished by this prefix, and there will be no risk of
> unintentional confusion.
> 
> Private types can start with the size of the prefix, and then
> organization can choose any prefix they like, or no prefix, if
> the size is of the prefix is 0 (means they are fine with possible
> conflicts with other empty-prefix private types)
> 

I don't think that should something that is required for people to do,
but perhaps it can be something that is strongly recommended and
suggested in the BIP itself.

> 
> Why not just say that the types should be encoded as 'compact size
> unsigned integer' ? This format for variable length integer encoding is
> already used in the BIP for other fields, and thus will not add any
> additional complexity to the parsing.
> 

On 7/31/19 10:32 AM, jan matejek via bitcoin-dev wrote:>
>
> why not use Bitcoin compact uint, which most PSBT consumers already
> implement?
>

There are a few issues with using a compact size uint. The main issue is
that it doesn't translate well to the proprietary use types. If we used
CSUint for the type, then all of type values for proprietary use need to
be reserved instead of allowing them to be infinitely expanded from the
initial set of proprietary use types.

There is also the fact that CSUints are malleable as the same value can
be represented in many different ways, just with different amounts of
leading zeroes. But I suppose that isn't really that big of an issue.

I am not opposed to using a CSUint, I just felt that it made things a
bit harder and was unnecessary.


Andrew

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility

[Andrew Chow via bitcoin-dev]

Hi All,

I would like to propose some types that allow for BIP 174 PSBT to be
extended more in the future.


Firstly, I would like to propose that some types be reserved for
proprietary use. These proprietary use types are, in general, for
private use by individuals and organizations who want to use PSBT in
their processes. These are usefule when there are additional data that
they need attached to a PSBT but such data are not useful (or available)
for the general public.

These types will be guaranteed to not be used by the public
specification and there is no expectation that any publicly available
software be able to understand any specific meanings of these types.
These types should be used for internal processes only.

The types I would like to reserve for proprietary use are the 15 types
from 0xF0 to 0xFE inclusive. These 15 type values will be the same for
global, per-input, and per-output types. If 15 types are not enough,
additional types can be obtained using the multi-byte type method
described later.


Next, I would like to propose a global version type and field. The
version type is 0xEF with only the type as the key, and the value is a
32-bit unsigned little endian integer representing the version number. A
PSBT without a version number is to be considered version 0. If a parser
sees a version number that it does not understand, it should exit
immediately as the PSBT will contain types that are not safe to ignore.

This version number is a safeguard in the event that a backwards
incompatible change is introduce to PSBT. While PSBT is designed and
intended to be forwards compatible by allowing parsers to ignore types
that they do not understand, it is possible that at ype is added in the
future which breaks this assumption and it would be unsafe for a type to
be ignored.

Updaters and combiners that need to add a version number to a PSBT
should use the highest version number required. For example, if a
combiner sees two PSBTs for the same transaction, one with version 0,
and the other with version 1, then it should combine them and produce a
PSBT with version 1. If an updater is updating a PSBT and needs to add a
field that is only available in version 1, then it should set the PSBT
version number to 1 unless a version higher than that is already specified.

It is not expected that the version number will ever be used. We try to
make PSBT fields safe to ignore. The version number is only being
included here as a safeguard in the event that breaking compatibilty is
required.


Lastly, I would like to propose the canonical method for mult-byte
types. We designate a specific type to indicate that the type is
multiple bytes. When such types are observed, parsers should move onto
the next byte and interpret that as the type, keeping in mind the number
of bytes that were read in for the type.

I propose that we use 0xFF as this designated type. When a parser sees
an 0xFF value as the type, it reads the next byte as being part of the
type. So two byte types will be of the form 0xFFXX. This method allows
us to do a prefix match in order to quickly identify the type being
used. For types with more bytes, simply use another 0xFF byte. So three
byte types would be of the form 0xXX, four byte, 0xFFXX, and so
on. When multi-byte types are specified in the BIP, they should be
specified in this full length form, i.e. two byte types as 0xFFXX.

The same mechanism can be used for the proprietary use types, just with
a different value as the designated multi-byte indicator. For example,
one could use 0xFE as the designated type as that is in the proprietary
types range. Of course any type within the proprietary type range could
be used as the indicator, it is up to the users to determine this
themselves.

While other methods of indicating multiple bytes and lengths may be more
space efficient and allow us to have more types represented in a smaller
space, I am choosing this method because of its simplicity. This is easy
to understand and implement. Furthermore, I do not expect that we will
use so many types. I don't think that we will need to have more than one
byte types for a very long time.


Please let me know your thoughts on these extensions. I will open a PR
to the bips repo to add these to BIP 174 if there are no objections.


Andrew Chow

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP174 amendment proposal (Important Signer Check should be mentioned)

[Andrew Chow via bitcoin-dev]

This was the original intent of the sighash field. Either the sighash is 
acceptable to the signer and the signer signs with it, or they do not sign at 
all.

On 7/9/19 11:58 AM, Jonathan Underwood via bitcoin-dev wrote:

> Hi all,
>
> Just to be brief, I'll kick off with an attack scenario.
>
> 1. I am a signer, I get a PSBT that is ready to sign. I parse. I sign 
> according to the PSBT as-is.
> 2. I notice my UTXO was stolen by a hacker because they changed my PSBT 
> input's sighashtype to SIGHASH_ANYONECANPAY | SIGHASH_NONE and after the fact 
> they changed the outputs to send to themselves, and added an input they 
> signed with SIGHASH_ALL.
> 3. I lose the BTC in my UTXO.
>
> So we should definitely add to the signer checks "ensure the sighash type 
> given is the type of sighash you want to sign." etc.
>
> My proposal for a wording change would be addition to the bullet list:
>
> - If a sighash type is provided, the signer MUST check that the sighash type 
> is acceptable to them, and fail signing if unacceptable.
> - If a sighash type is not provided, the signer SHOULD sign using 
> SIGHASH_ALL, but may sign with any sighash type they wish.
>
> Any thoughts?
>
> Thanks,
> Jon
>
> --
>
> -
> Jonathan Underwood
> ビットバンク社　チーフビットコインオフィサー
> -
>
> 暗号化したメッセージをお送りの方は下記の公開鍵をご利用下さい。
>
> 指紋: 0xCE5EA9476DE7D3E45EBC3FDAD998682F3590FEA3___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Adding xpub field to PSBT to make multisig more secure

[Andrew Chow via bitcoin-dev]

Hi Stepan,

I think that this would be a good extension.

Just for clairty, by xpub, do you mean the extended serialization format 
defined in BIP 32 or the Base58 check encoded string of that serialization?

Andrew

On 4/26/19 11:21 AM, Stepan Snigirev via bitcoin-dev wrote:
> Hi list,
>
> I was looking at the bip174 PSBT specs, in particular for 
> multisignature setup, and I think with current spec there is a way to 
> steal user funds in M of N setup with M ≤ N/2.
>
> I made a small write-up on this: 
> https://github.com/stepansnigirev/random_notes/blob/master/psbt_multisig.md
>
> To compress:
>
> Currently in PSBT there is no way to reliably say if the output uses 
> the keys derived from the same root keys as the inputs aside from the 
> key owned by the signer => there is no way to verify that the output 
> is a change output in multisig setup.
>
> Therefore an attacker can replace half of the keys in the change 
> address by his own keys and still get the transaction signed.
>
> I suggest to add an xpub field to the inputs and outputs metadata, 
> then signers can verify that the same xpubs are used for public keys 
> in inputs and outputs => output is indeed a change.
>
> Normally change and receiving addresses are derived from the same xpub 
> with non-hardened derivation pathes, so providing xpub after the last 
> hardened index should be enough to see that public keys of inputs and 
> change output are derived from the same xpub.
>
> I suggest to add the following key-value pairs to PSBT:
>
> Type: BIP 32 public key `PSBT_IN_BIP32_XPUB = 0x10`
> - Key: derivation path for xpub
>   `{0x10}|{master key fingerprint}|{32-bit int}|...|{32-bit int}`
> - Value: 78-byte xpub value
>   `{xpub}`
>
> Type: BIP 32 public key `PSBT_OUT_BIP32_XPUB = 0x03`
> - Key: derivation path for xpub
>   `{0x03}|{master key fingerprint}|{32-bit int}|...|{32-bit int}`
> - Value: 78-byte xpub value
>   `{xpub}`
>
> Derivation paths are in the key of the key-value pair as they are used 
> for lookup, and xpub itself is the actual value being looked up.
>
> I also want to mention that Trezor for example doesn't suffer from 
> this problem as they use xpubs to verify change outputs. So it may 
> make sense to go through the communication protocols of existing 
> hardware / multisignature wallets and see if there is something else 
> we are missing.
>
> If everyone is happy about the proposal I would prepare a pull request 
> to the bip.
>
> Best regards,
> Stepan Snigirev.
>

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Using the same public keys, the p2sh returned by `addmultisigaddress` differs from the one returned by `createmultisigaddress`

[Andrew Chow via bitcoin-dev]

Hi Michele,

You are seeing this discrepancy due to the address types in use. 
addmultisigaddress uses the default address type of the wallet, which is 
p2sh-segwit. createmultisig uses a default address type of legacy. To have 
createmultisig get addmultisigaddress's result, you need to add the string 
"p2sh-segwit" to the end of your command. To have addmultisigaddress get 
createmultisig's result, you need to add the string "legacy" to the end of your 
command.

On 4/19/19 7:53 AM, Michele Federici via bitcoin-dev wrote:

> Hi everyone,
>
> I'm writing here because I didn't find any resources in the docs or
> somewhere else online explaining this, I don't get if this is a bug or
> I'm missing something.
>
> I was working on a function to derive the pay-to-script-hash from a
> multisig script and I was checking the results against the bitcoin
> core's `addmultisigaddress` output, although I was quite sure that my
> implementation was correct, my output address was different.
> By chance, I then tried the `createmultisigaddress` method, using the
> same public keys, and this time the output was matching with mine.
>
> I thought the outputs of `addmultisigaddress` and
> `createmultisigaddress` were supposed to be the same, but instead are
> inconsistent from each other:
>
> ```
> bitcoin-cli addmultisigaddress 1
> '["045897fee25bd7c5692510b2f50fcae9aa20fbc4d49d59814f4c7fdb5c4bc6eb1c0c
> e382458f9588e922e0d509ed8d34856787380075b00418b02e0bf7c652ef9d","02ac46
> c6d74d15e60f4f1035ff07ef740aca1d68d55ba0b8d336a73d7a35858831","0224a4dc
> 5620714a9ecf67a09583d1e4c04f5bedb8ecea99028da05bb15a2a7e07"]'
> {
> "address": "36ULucjWUTrDvaJzCyhFoVbDoNS6Zum2Du",
> "redeemScript":
> "5141045897fee25bd7c5692510b2f50fcae9aa20fbc4d49d59814f4c7fdb5c4bc6eb1c
> 0ce382458f9588e922e0d509ed8d34856787380075b00418b02e0bf7c652ef9d2102ac4
> 6c6d74d15e60f4f1035ff07ef740aca1d68d55ba0b8d336a73d7a35858831210224a4dc
> 5620714a9ecf67a09583d1e4c04f5bedb8ecea99028da05bb15a2a7e0753ae"
> }
> ```
>
> ```
> bitcoin-cli createmultisig 1
> '["045897fee25bd7c5692510b2f50fcae9aa20fbc4d49d59814f4c7fdb5c4bc6eb1c0c
> e382458f9588e922e0d509ed8d34856787380075b00418b02e0bf7c652ef9d","02ac46
> c6d74d15e60f4f1035ff07ef740aca1d68d55ba0b8d336a73d7a35858831","0224a4dc
> 5620714a9ecf67a09583d1e4c04f5bedb8ecea99028da05bb15a2a7e07"]'
> {
> "address": "3GiimyxF1R5VixfBFAbQZbuy9EesD2r6n1",
> "redeemScript":
> "5141045897fee25bd7c5692510b2f50fcae9aa20fbc4d49d59814f4c7fdb5c4bc6eb1c
> 0ce382458f9588e922e0d509ed8d34856787380075b00418b02e0bf7c652ef9d2102ac4
> 6c6d74d15e60f4f1035ff07ef740aca1d68d55ba0b8d336a73d7a35858831210224a4dc
> 5620714a9ecf67a09583d1e4c04f5bedb8ecea99028da05bb15a2a7e0753ae"
> }
> ```
>
> I was also pretty confused by the fact that the `redeemScript` is the
> same, only the addresses are different, and calling `decodescript` with
> it I get the same address as `createmultisig`:
>
> ```
> bitcoin-cli decodescript
> "5141045897fee25bd7c5692510b2f50fcae9aa20fbc4d49d59814f4c7fdb5c4bc6eb1c
> 0ce382458f9588e922e0d509ed8d34856787380075b00418b02e0bf7c652ef9d2102ac4
> 6c6d74d15e60f4f1035ff07ef740aca1d68d55ba0b8d336a73d7a35858831210224a4dc
> 5620714a9ecf67a09583d1e4c04f5bedb8ecea99028da05bb15a2a7e0753ae"
> {
> "asm": "1
> 045897fee25bd7c5692510b2f50fcae9aa20fbc4d49d59814f4c7fdb5c4bc6eb1c0ce38
> 2458f9588e922e0d509ed8d34856787380075b00418b02e0bf7c652ef9d
> 02ac46c6d74d15e60f4f1035ff07ef740aca1d68d55ba0b8d336a73d7a35858831
> 0224a4dc5620714a9ecf67a09583d1e4c04f5bedb8ecea99028da05bb15a2a7e07 3
> OP_CHECKMULTISIG",
> "reqSigs": 1,
> "type": "multisig",
> "addresses": [
> "12PfkcWheYsfFddWfHhaXpFDVx78gnKQ9k",
> "1AYLXzXd6N2avqW4j8Gyhb8jb2jXvNPyuV",
> "1PWsxtcBMRHTSX2L7wrXgwnFigHD3KhbFT"
> ],
> "p2sh": "3GiimyxF1R5VixfBFAbQZbuy9EesD2r6n1"
> }
> ```
>
> I don't understand, how can this be possible?
>
> Thank you,
> Michele
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP174 / PSBT extensions

[Andrew Chow via bitcoin-dev]

Hi Andrew,

I think having some of these extensions would be great.

On 3/6/19 1:08 PM, Andrew Poelstra via bitcoin-dev wrote:

> 1. Add an field to PSBT_GLOBAL_UNSIGNED_TX to the global table which contains
>just a txid of the unsigned transaction, for bandwidth savings in case
>signers have already seen the tx or can construct it themselves.
>
>This field would be fixed 32 bytes.
>
>(This would actually be a breaking change since the current PSBT rules 
> require
>PSBT_GLOBAL_UNSIGNED_TX to always be present. Maybe this is a no-go for 
> that
>reason alone.)

I feel like this breaks the central idea of PSBT that a PSBT contains 
everything you need to construct a transaction.
This would rely on parties in the transaction having state and remembering 
things which I don't think is something
that we can assume.

> 2. Add a version field to the global table.

For what purpose?

The rest of the proposed extensions I think are fine.

Regards,

Andrew Chow

> 3. Add fields to the per-input tables for
>(a) confirmed depth of the referenced txout; this is useful for finalizers
>trying to create optimized witnesses, for e.g. cases when CSV timeouts
>expire and some signatures become unnecessary.
>
>This field must be a varint.
>
>(b) a map from SHA2 hashes to their 32-byte preimages; this field must be
>fixed 32 bytes. This, plus the CSV thing, would allow writing 
> finalizers
>that work with all of Miniscript [2].
>
>(c) a map from public keys to 32-byte "tweaks" that are used in the 
> pay-to-contract
>construction. Selfishly I'd like this to be a variable-length 
> bytestring
>with the semantics that (a) the first 33 bytes represent an untweaked
>pubkey; (b) the HMAC-SHA256 of the whole thing, when multiplied by G 
> and
>added to the untweaked pubkey, result in the target key. This matches 
> the
>algorithm in [3] which is deployed in Blockstream's Liquid, but I'd be
>happy with a more efficient scheme which e.g. used SHA256 rather than
>HMAC-SHA256.
>
>(d) maps from public keys to partial nonce commitments, partial nonces, and
>partial signatures, for MuSig [4] support.
>
>(e) a map from signatures (or signature nonces?) to sign-to-contract 
> tweaks.
>Same semantics as (c) above.
>
>The last two suggestions are probably premature and need further 
> development
>and standardization of the related protocols. But I'm throwing them in to 
> see
>if other people have strong feelings about this.
>
> 4. Add fields to the per-output tables for pay-to-contract, like in (c) above.
>
> 5. Add a field (or rather, family of fields) to every table which is 
> "proprietary
>use" and guaranteed not to be defined by any future PSBT extension. 
> Specifically
>every field with key-type 0xFF could be considered "proprietary".
>
> 5a. The special field in the global table whose key is only 0xFF should be a
> "proprietary version field" with unspecified semantics but an 
> understanding
> that specific users might stick a GUID or something in there as a way to
> recognize their own PSBTs.
>
> [1]
> https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki#Encoding
> [2]
> http://bitcoin.sipa.be/miniscript/miniscript.html
> [3]
> https://github.com/ElementsProject/elements/blob/elements-0.14.1/src/validation.cpp
> [4]
> https://eprint.iacr.org/2018/068
> --
> Andrew Poelstra
> Director of Research, Blockstream
> Email: apoelstra at wpsoftware.net
> Web:
> https://www.wpsoftware.net/andrew
> "There are some mornings when the sky looks like a road
>  There are some dragons who were built to have and hold
>  And some machines are dropped from great heights lovingly
>  And some great bellies ache with many bumblebees
>  And they sting so terribly"
>--Joanna Newsom
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

>___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [bitcoin-core-dev] Bitcoin Core update notice

[Andrew Chow via bitcoin-dev]

The backported versions have not been released yet. They are still going
through the gitian build process. 0.16.3 was the first one to be
released so that is the one that everyone is being recommended to
upgrade to. Regardless, you should upgrade to a patched version, whether
that is 0.14.3, 0.15.2, or 0.16.3. It is not misinformation that
everybody must upgrade.


On 09/21/2018 06:39 PM, gb via bitcoin-dev wrote:
> If the bugfix can be backported to earlier versions why is the
> hype/hysteria about "everybody" must immediately upgrade to 0.16.3
> currently being spread on the forums/reddit?
>
> https://bitcointalk.org/index.php?topic=5034070.0
> https://old.reddit.com/r/Bitcoin/comments/9hp90p/1775_nodes_out_of_9616_185_are_currently_on/
>
> I don't see any effort to correct this misinformation either.
>
> Regards.
>
> On Tue, 2018-09-18 at 17:06 -0700, Pieter Wuille via bitcoin-core-dev
> wrote:
>> Hello all,
>>
>> Bitcoin Core 0.16.3 was just released with a fix for
>> CVE-2018-17144:
>> https://bitcoincore.org/en/2018/09/18/release-0.16.3/
>>
>> We urge all network participants to upgrade to 0.16.3[*] as soon
>> as possible.
>>
>> [*] For those who build from source, the 0.14, 0.15, 0.16, 0.17,
>> and master branches on GitHub (https://github.com/bitcoin/bitcoin)
>> are fixed as well.
>>
>> --
>> Pieter
>> ___
>> bitcoin-core-dev mailing list
>> bitcoin-core-...@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-core-dev
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 174 thoughts

[Andrew Chow via bitcoin-dev]

I disagree with the idea that global types can be removed. Firstly, it
is less of a breaking change to leave it there than to remove it
entirely. Secondly, there may be a point in the future where global
types would be useful/necessary. By having it still be there, we allow
for future extensibility.

Andrew


On 06/24/2018 01:19 AM, Andrea wrote:
> Hi, 
>
> I think in the revised spec we can remove completely the "global types" as a 
> map or even as typed record. Since there is only one type (the transaction) 
> and it's compulsory to have one (and only one) we could just drop the 
> definition of global type and the key associated with it, simply after the 
> header + separator there must be a transaction.​​ Having read all the 
> discussion i also agree having per-input key derivation and per-output data 
> is a lot more handy for signers, no special feeling regarding the 
> encoding.Looking forward for the test vectors and the new spec.
>
> Cheers, Andrea.
>
> ‐‐‐ Original Message ‐‐‐
>
> On June 23, 2018 10:33 PM, Andrew Chow via bitcoin-dev 
>  wrote:
>
>> ​​
>>
>> On 06/23/2018 10:00 AM, William Casarin wrote:
>>
>>> Since we're still considering the encoding, I wonder if it would be a
>>>
>>> good idea to have a human-readible part like lightning invoices[1]?
>> I don't think that is necessary.
>>
>>> Then perhaps you could drop the magic code as well?
>> The magic is still necessary for the binary format in order to prevent
>>
>> normal transaction deserializers from accidentally deserializing a psbt.
>>
>>> Also we could do a base encoding that excludes + and / characters, such
>>>
>>> as base62 (gmp-style). It's easier to copy/paste (double clicking a
>>>
>>> string stops at / or + in base64 encodings).
>> While that would be ideal, I think it is better to use an encoding that
>>
>> most wallets already support. Most wallets already have Base64 decoding
>>
>> available so that they can decode signed messages which also use Base64
>>
>> encoding. I think it is unnecessary to introduce another encoding.
>>
>> On 06/23/2018 11:27 AM, Peter D. Gray wrote:
>>
>>> Personally, I don't think you should spec an encoding. It should be binary 
>>> only and hex for developers and JSON interfaces. My thinking is that PSBT's 
>>> are not user-visible things. They have a short lifetime and are nothing 
>>> something that is "stored" or referenced much later. Hex is good enough and 
>>> has no downsides as an excoding except for density.
>> I think what will end up happening though is that, at least in the
>>
>> beginning, PSBTs will primarily be strings that people end up copy and
>>
>> pasting. Since a PSBT can get pretty large, the strings are rather
>>
>> cumbersome to move around, especially as hex. At least with Base64 the
>>
>> strings will be smaller.
>>
>>> On the other hand, suggesting a filename extension (probably .PSBT?) and a 
>>> mime-type to match, are helpful since wallets and such will want to 
>>> register with their operating systems to become handlers of those 
>>> mimetypes. Really that's a lot more important for interoperability at this 
>>> point, than an encoding.
>> Agreed. I will include those in the BIP.
>>
>>> Looking forward to test vectors, and I might have more to say once my code 
>>> can handle them (again).
>>>
>>> Feedback on the BIP as it stands now:
>>>
>>> -   Appendix A needs an update, and I suggest defining symbols 
>>> (PK_PARTIAL_SIG == 0x02) for the numeric key values. This helps 
>>> implementers as we don't all define our own symbols and/or use numeric 
>>> constants in our code.
>> Okay.
>>
>>> -   Those tables are just not working. Might want to reformat as 
>>> descriptive lists, point form, or generally anything else... sorry.
>> I will try my best to fix that. Mediawiki sucks...
>>
>> Andrew
>>
>> bitcoin-dev mailing list
>>
>> bitcoin-dev@lists.linuxfoundation.org
>>
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP 174 thoughts

[Andrew Chow via bitcoin-dev]

On 06/23/2018 10:00 AM, William Casarin wrote:
> Since we're still considering the encoding, I wonder if it would be a
> good idea to have a human-readible part like lightning invoices[1]?
I don't think that is necessary.
> Then perhaps you could drop the magic code as well?
The magic is still necessary for the binary format in order to prevent
normal transaction deserializers from accidentally deserializing a psbt.
> Also we could do a base encoding that excludes + and / characters, such
> as base62 (gmp-style). It's easier to copy/paste (double clicking a
> string stops at / or + in base64 encodings).
While that would be ideal, I think it is better to use an encoding that
most wallets already support. Most wallets already have Base64 decoding
available so that they can decode signed messages which also use Base64
encoding. I think it is unnecessary to introduce another encoding.


On 06/23/2018 11:27 AM, Peter D. Gray wrote:
> Personally, I don't think you should spec an encoding. It should be binary 
> only and hex for developers and JSON interfaces. My thinking is that PSBT's 
> are not user-visible things. They have a short lifetime and are nothing 
> something that is "stored" or referenced much later. Hex is good enough and 
> has no downsides as an excoding except for density.
I think what will end up happening though is that, at least in the
beginning, PSBTs will primarily be strings that people end up copy and
pasting. Since a PSBT can get pretty large, the strings are rather
cumbersome to move around, especially as hex. At least with Base64 the
strings will be smaller.
> On the other hand, suggesting a filename extension (probably .PSBT?) and a 
> mime-type to match, are helpful since wallets and such will want to register 
> with their operating systems to become handlers of those mimetypes. Really 
> that's a lot more important for interoperability at this point, than an 
> encoding.
Agreed. I will include those in the BIP.
> Looking forward to test vectors, and I might have more to say once my code 
> can handle them (again).
>
> Feedback on the BIP as it stands now: 
>
> - Appendix A needs an update, and I suggest defining symbols (PK_PARTIAL_SIG 
> == 0x02) for the numeric key values. This helps implementers as we don't all 
> define our own symbols and/or use numeric constants in our code.
Okay.
> - Those tables are just not working. Might want to reformat as descriptive 
> lists, point form, or generally anything else... sorry.
I will try my best to fix that. Mediawiki sucks...

Andrew

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] version.relay behavior change

[Andrew Chow via bitcoin-dev]

I don't think the nodes that you are connecting to that have this
behavior are actually forked from Bitcoin Core. It seems more like fake
nodes - nodes that don't actually do any verification or follow the
protocol. Such fake nodes can set whatever user agent they want, common
ones being Bitcoin Core's user agents.

IMO your best solution would be to drop peers for protocol noncompliance.

Andrew


On 03/15/2018 05:17 AM, Eric Voskuil wrote:
> Thanks for the reply Andrew. I’ve reviewed the relevant Core sources
> and I do not see any problem. We have also synced against a Core node
> locally and not seen the problem.
>
> The reason I suspected it was Core is that it is very common and all
> of the User Agents are consistent (with an occasional exception for
> forked nodes). So there’s no easy way to determine what sort of nodes
> we are seeing. 
>
> We tend to cycle through many more connections during sync than a Core
> node, so may just be seeing it more frequently, but I assume Core
> would log this behavior as well. Even so, seeing that wouldn’t help
> much. I’m as certain as I can be at this point that we are setting the
> flag and version correctly (and that we do not set bip37 filters).
>
> This behavior started infrequently with 0.14.0 peers and has become
> more common over time. Just wondering at this point what fork would
> report as Core and be that common? We used to drop peers that did this
> (for protocol noncompliance), and I’m considering reinstating that
> behavior.
>
> e
>
> On Mar 9, 2018, at 16:33, Andrew Chow via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org
> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>
>> Looking through the code, I don't think that this behavior has
>> changed. Are you sure that you are actually connected to
>> Satoshi:0.15.0 nodes and not a node that has simply set their
>> user-agent to that (i.e. not a real Satoshi:0.15.0 node)?
>>
>> If what you are seeing is true, it is likely a bug and not an
>> intentional change. In that case, can you provide specific details on
>> how to reproduce?
>>
>> Andrew
>>
>>
>> On 03/09/2018 02:50 AM, Eric Voskuil via bitcoin-dev wrote:
>>> /Satoshi:0.15.0/ and later nodes appear to be no longer honoring the
>>> version.relay=false flag (BIP37). Could someone familiar with the change
>>> please explain the rational?
>>>
>>> Thanks,
>>>
>>> e
>>>
>>>
>>>
>>> ___
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>> ___
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> <mailto:bitcoin-dev@lists.linuxfoundation.org>
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] version.relay behavior change

[Andrew Chow via bitcoin-dev]

Looking through the code, I don't think that this behavior has changed.
Are you sure that you are actually connected to Satoshi:0.15.0 nodes and
not a node that has simply set their user-agent to that (i.e. not a real
Satoshi:0.15.0 node)?

If what you are seeing is true, it is likely a bug and not an
intentional change. In that case, can you provide specific details on
how to reproduce?

Andrew


On 03/09/2018 02:50 AM, Eric Voskuil via bitcoin-dev wrote:
> /Satoshi:0.15.0/ and later nodes appear to be no longer honoring the
> version.relay=false flag (BIP37). Could someone familiar with the change
> please explain the rational?
>
> Thanks,
>
> e
>
>
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] [BIP Proposal] Partially Signed Bitcoin Transaction (PSBT) format

[Andrew Chow via bitcoin-dev]

Hi everyone,

I would like to propose a standard format for unsigned and partially signed
transactions.

===Abstract===

This document proposes a binary transaction format which contains the
information
necessary for a signer to produce signatures for the transaction and holds
the
signatures for an input while the input does not have a complete set of
signatures.
The signer can be offline as all necessary information will be provided in
the
transaction.

===Motivation===

Creating unsigned or partially signed transactions to be passed around to
multiple
signers is currently implementation dependent, making it hard for people
who use
different wallet software from being able to easily do so. One of the goals
of this
document is to create a standard and extensible format that can be used
between clients to allow
people to pass around the same transaction to sign and combine their
signatures. The
format is also designed to be easily extended for future use which is
harder to do
with existing transaction formats.

Signing transactions also requires users to have access to the UTXOs being
spent. This transaction
format will allow offline signers such as air-gapped wallets and hardware
wallets
to be able to sign transactions without needing direct access to the UTXO
set and without
risk of being defrauded.

The full text can be found here:
https://github.com/achow101/bips/blob/bip-psbt/bip-psbt.mediawiki

Andrew Chow
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] A Segwit2x BIP

[Andrew Chow via bitcoin-dev]

What's special about block 475776?


On July 13, 2017 12:23:46 PM Sergio Demian Lerner via bitcoin-dev 
 wrote:



The BIP has been updated.

Changes:
- The technical spec has been improved: now the block size increase is
specified in terms of weight and not in terms of bytes.
- The increase in the maximum block sigops after HF has been documented.
- Comments added about the worst case block size.

Happy weekend! And don't forget to start signaling something before block
475776 !  It's just 90 blocks away.
Bit 1 or 4,1 or whatever you wish, but please signal something.

To the moon!


On Wed, Jul 12, 2017 at 2:38 PM, Jorge Timón via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:




On 12 Jul 2017 2:31 pm, "Tom Zander via bitcoin-dev"  wrote:

On Monday, 10 July 2017 20:38:08 CEST Jorge Timón via bitcoin-dev wrote:
> I think anything less than 1 year after release of tested code by some
> implementation would be irresponsible for any hardfork, even a very
> simple one.

Good news!

Code to support 2x (the hard fork part of the proposal) has been out and
tested for much longer than that.


Not true. It's different code on top of segwit. The first attempt in btc1
(very recent) didn't even increased the size (because it changed the
meaningless "base size" without touching the weight limit. As for the
current code, I don't think it has been properly tested today, let alone
"for mucj longer than 1 year.
Anyway, I said, one year from tested release. Segwitx2 hasn't been
released, has it? If so, too late to discuss a bip imo, the bip may end up
being different from what has been released due to feedback (unless it is
ignored again, of course).


--
Tom Zander
Blog: https://zander.github.io
Vlog: https://vimeo.com/channels/tomscryptochannel
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev



___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev






--
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Reduced signalling threshold activation of existing segwit deployment

[Andrew Chow via bitcoin-dev]

Ah. I see now. It wasn't very clear to me that that is what will happen.

Also, shouldn't the timeout date be set for before the BIP141 timeout?
Otherwise this could lock in but not have enough time for Segwit to be
locked in.


On 5/23/2017 4:42 PM, James Hilliard wrote:
> That is incorrect, it is compatible with the current segwit
> implementation because it triggers a mandatory signalling period that
> will activate segwit on existing nodes.
>
> On Tue, May 23, 2017 at 4:39 PM, Andrew Chow via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
>> Hi James,
>>
>> From what I understand, this proposal is incompatible with the current
>> segwit implementation with regards to the NODE_WITNESS service bit. I
>> believe it could cause network partitioning if the service bit is not
>> changed.
>>
>> Andrew
>>
>>
>> On 5/22/2017 6:40 PM, James Hilliard via bitcoin-dev wrote:
>>> I would like to propose an implementation that accomplishes the first
>>> part of the Barry Silbert proposal independently from the second:
>>>
>>> "Activate Segregated Witness at an 80% threshold, signaling at bit 4"
>>> in a way that
>>>
>>> The goal here is to minimize chain split risk and network disruption
>>> while maximizing backwards compatibility and still providing for rapid
>>> activation of segwit at the 80% threshold using bit 4.
>>>
>>> By activating segwit immediately and separately from any HF we can
>>> scale quickly without risking a rushed combined segwit+HF that would
>>> almost certainly cause widespread issues.
>>>
>>> Draft proposal:
>>> https://github.com/jameshilliard/bips/blob/bip-segsignal/bip-segsignal.mediawiki
>>>
>>> Proposal text:
>>> 
>>>   BIP: segsignal
>>>   Layer: Consensus (soft fork)
>>>   Title: Reduced signalling threshold activation of existing segwit 
>>> deployment
>>>   Author: James Hilliard <james.hillia...@gmail.com>
>>>   Status: Draft
>>>   Type: Standards Track
>>>   Created: 2017-05-22
>>>   License: BSD-3-Clause
>>>CC0-1.0
>>> 
>>>
>>> ==Abstract==
>>>
>>> This document specifies a method to activate the existing BIP9 segwit
>>> deployment with a majority hashpower less than 95%.
>>>
>>> ==Definitions==
>>>
>>> "existing segwit deployment" refer to the BIP9 "segwit" deployment
>>> using bit 1, between November 15th 2016 and November 15th 2017 to
>>> activate BIP141, BIP143 and BIP147.
>>>
>>> ==Motivation==
>>>
>>> Segwit increases the blocksize, fixes transaction malleability, and
>>> makes scripting easier to upgrade as well as bringing many other
>>> [https://bitcoincore.org/en/2016/01/26/segwit-benefits/ benefits].
>>>
>>> This BIP provides a way for a simple majority of miners to coordinate
>>> activation of the existing segwit deployment with less than 95%
>>> hashpower. For a number of reasons a complete redeployment of segwit
>>> is difficulty to do until the existing deployment expires. This is due
>>> to 0.13.1+ having many segwit related features active already,
>>> including all the P2P components, the new network service flag, the
>>> witness-tx and block messages, compact blocks v2 and preferential
>>> peering. A redeployment of segwit will need to redefine all these
>>> things and doing so before expiry would greatly complicate testing.
>>>
>>> ==Specification==
>>>
>>> While this BIP is active, all blocks must set the nVersion header top
>>> 3 bits to 001 together with bit field (1<<1) (according to the
>>> existing segwit deployment). Blocks that do not signal as required
>>> will be rejected.
>>>
>>> ==Deployment==
>>>
>>> This BIP will be deployed by a "version bits" with an 80%(this can be
>>> adjusted if desired) activation threshold BIP9 with the name
>>> "segsignal" and using bit 4.
>>>
>>> This BIP will have a start time of midnight June 1st, 2017 (epoch time
>>> 1496275200) and timeout on midnight November 15th 2017 (epoch time
>>> 1510704000). This BIP will cease to be active when segwit is
>>> locked-in.
>>>
>>> === Reference implementation ===
>>>
>>> 
>>> // Check if Segregated Witness is Locked In
>>> bool IsWitnessLockedIn(const CBlockIndex* pindexPrev, con

Re: [bitcoin-dev] Reduced signalling threshold activation of existing segwit deployment

[Andrew Chow via bitcoin-dev]

Hi James,

>From what I understand, this proposal is incompatible with the current
segwit implementation with regards to the NODE_WITNESS service bit. I
believe it could cause network partitioning if the service bit is not
changed.

Andrew


On 5/22/2017 6:40 PM, James Hilliard via bitcoin-dev wrote:
> I would like to propose an implementation that accomplishes the first
> part of the Barry Silbert proposal independently from the second:
>
> "Activate Segregated Witness at an 80% threshold, signaling at bit 4"
> in a way that
>
> The goal here is to minimize chain split risk and network disruption
> while maximizing backwards compatibility and still providing for rapid
> activation of segwit at the 80% threshold using bit 4.
>
> By activating segwit immediately and separately from any HF we can
> scale quickly without risking a rushed combined segwit+HF that would
> almost certainly cause widespread issues.
>
> Draft proposal:
> https://github.com/jameshilliard/bips/blob/bip-segsignal/bip-segsignal.mediawiki
>
> Proposal text:
> 
>   BIP: segsignal
>   Layer: Consensus (soft fork)
>   Title: Reduced signalling threshold activation of existing segwit deployment
>   Author: James Hilliard 
>   Status: Draft
>   Type: Standards Track
>   Created: 2017-05-22
>   License: BSD-3-Clause
>CC0-1.0
> 
>
> ==Abstract==
>
> This document specifies a method to activate the existing BIP9 segwit
> deployment with a majority hashpower less than 95%.
>
> ==Definitions==
>
> "existing segwit deployment" refer to the BIP9 "segwit" deployment
> using bit 1, between November 15th 2016 and November 15th 2017 to
> activate BIP141, BIP143 and BIP147.
>
> ==Motivation==
>
> Segwit increases the blocksize, fixes transaction malleability, and
> makes scripting easier to upgrade as well as bringing many other
> [https://bitcoincore.org/en/2016/01/26/segwit-benefits/ benefits].
>
> This BIP provides a way for a simple majority of miners to coordinate
> activation of the existing segwit deployment with less than 95%
> hashpower. For a number of reasons a complete redeployment of segwit
> is difficulty to do until the existing deployment expires. This is due
> to 0.13.1+ having many segwit related features active already,
> including all the P2P components, the new network service flag, the
> witness-tx and block messages, compact blocks v2 and preferential
> peering. A redeployment of segwit will need to redefine all these
> things and doing so before expiry would greatly complicate testing.
>
> ==Specification==
>
> While this BIP is active, all blocks must set the nVersion header top
> 3 bits to 001 together with bit field (1<<1) (according to the
> existing segwit deployment). Blocks that do not signal as required
> will be rejected.
>
> ==Deployment==
>
> This BIP will be deployed by a "version bits" with an 80%(this can be
> adjusted if desired) activation threshold BIP9 with the name
> "segsignal" and using bit 4.
>
> This BIP will have a start time of midnight June 1st, 2017 (epoch time
> 1496275200) and timeout on midnight November 15th 2017 (epoch time
> 1510704000). This BIP will cease to be active when segwit is
> locked-in.
>
> === Reference implementation ===
>
> 
> // Check if Segregated Witness is Locked In
> bool IsWitnessLockedIn(const CBlockIndex* pindexPrev, const
> Consensus::Params& params)
> {
> LOCK(cs_main);
> return (VersionBitsState(pindexPrev, params,
> Consensus::DEPLOYMENT_SEGWIT, versionbitscache) ==
> THRESHOLD_LOCKED_IN);
> }
>
> // SEGSIGNAL mandatory segwit signalling.
> if ( VersionBitsState(pindex->pprev, chainparams.GetConsensus(),
> Consensus::DEPLOYMENT_SEGSIGNAL, versionbitscache) == THRESHOLD_ACTIVE
> &&
>  !IsWitnessLockedIn(pindex->pprev, chainparams.GetConsensus()) &&
> // Segwit is not locked in
>  !IsWitnessEnabled(pindex->pprev, chainparams.GetConsensus()) ) //
> and is not active.
> {
> bool fVersionBits = (pindex->nVersion & VERSIONBITS_TOP_MASK) ==
> VERSIONBITS_TOP_BITS;
> bool fSegbit = (pindex->nVersion &
> VersionBitsMask(chainparams.GetConsensus(),
> Consensus::DEPLOYMENT_SEGWIT)) != 0;
> if (!(fVersionBits && fSegbit)) {
> return state.DoS(0, error("ConnectBlock(): relayed block must
> signal for segwit, please upgrade"), REJECT_INVALID, "bad-no-segwit");
> }
> }
> 
>
> https://github.com/bitcoin/bitcoin/compare/0.14...jameshilliard:segsignal-v0.14.1
>
> ==Backwards Compatibility==
>
> This deployment is compatible with the existing "segwit" bit 1
> deployment scheduled between midnight November 15th, 2016 and midnight
> November 15th, 2017. Miners will need to upgrade their nodes to
> support segsignal otherwise they may build on top of an invalid block.
> While this bip is active users should either upgrade to segsignal or
> wait for additional confirmations when accepting payments.
>
> ==Rationale==
>
> Historically we have used IsSuperMajority() to activate soft forks
> such as BIP66 which has a 

Re: [bitcoin-dev] Issolated Bitcoin Nodes

[Andrew Chow via bitcoin-dev]

The issue is due to Segwit blocks since Testnet has already activated
Segwit. 0.12.x- nodes will receive a Segwit block with all of the
witnesses stripped. When they relay this block to a 0.13.0+ node, the
block will be rejected because those have Segwit functionality and
require the witnesses to be in the block. Given that Testnet has a
smaller number of nodes and less difficulty, this could result in some
miners using 0.13.0+ mining blocks which do not propagate well and thus
causing multiple chain splits and reorgs as other miners find blocks for
the same height before receiving a block for that height.


On 3/23/2017 6:37 PM, Juan Garavaglia via bitcoin-dev wrote:
>
> We notice some reorgs in Bitcoin testnet, while reorgs in testnet are
> common and may be part of different tests and experiments, it seems
> the forks are not created by a single user and multiple blocks were
> mined by different users in each chain.  My first impression was that
> the problem was related to network issues but some Bitcoin explorers
> were following one chain while others follow the other one. 
> Nonetheless, well established explorers like blocktrail.com or
> blockr.io were following different chains at different heights which
> led to me to believe that it was not a network issue. After some time,
> a reorg occurs and it all comes to normal state as a single chain.
>
> We started investigating more and we identified that the fork occurs
> with nodes 0.12; in some situations, nodes 0.12 has longer/different
> chains. The blocks in both chains are valid so something must be
> occurring in the communication between nodes but not related with the
> network itself.
>
> Long story short, when nodes 0.13+ receive blocks from 0.13+ nodes all
> is ok, and those blocks propagate to older nodes with no issues. But
> when a block tries to be propagated from bitcoind 0.12.+ to newer ones
> those blocks are NOT being propagated to the peers with newer versions
> while these newer blocks are being propagated to peers with older
> versions with no issues.
>
> My conclusion is that we have a backward compatibility issue between
> 0.13.X+ and older versions.
>
> The issue is simple to replicate, first, get latest version of
> bitcoind, complete the IBD after is at current height, then force it
> to use exclusively one or more peers of versions 0.12.X and older, and
> you will notice that the latest version node will never receive a new
> block.
>
> Probably some alternative bitcoin implementations act as bridges
> between these two versions and facilitate the chain reorgs.
>
> I have not yet found any way where/how it can be used in a malicious
> way or be exploited by a miner but in theory Bitcoin 0.13.X+ should
> remain compatible with older ones, but a 0.13+ node may become
> isolated by 0.12 peers, and there is not notice for the node owner.
>
>  
>
>
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] High consensus fork system for scaling without limits

[Andrew Chow via bitcoin-dev]

Hi Erik,

I have left you some comments below.

Some general questions:
How will you deal with excessive sighashing (i.e. massive transactions
that include a lot of signature verification)?
Presumably the sigops limit will increase proportionally?

On 3/8/2017 2:42 PM, Erik Aronesty via bitcoin-dev wrote:
> I woudl like to propose a BIP that works something like this:
>
> 1. Allow users to signal readiness by publishing an EB. This EB is an
> absolute upper bound, and cannot be overridden by miners. Current EB
> is 1MB, the status-quo.   Maybe EB can be configured in a config file,
> not a UI, since it's an "advanced" feature.
What does EB stand for?

What is the point of users publishing an EB? Is it for miners to
determine what to set theirs to? If so, what about sybil attacks with
fake nodes publishing EBs?

How do users publish an EB? Do they use a transaction? Or is it
something that goes into the User Agent?

How high can the EB go? What is its maximum?
> 2. Miners can also signal readiness by publishing their own EB in a block.
>
> 3. If 95% of blocks within a one month signalling period contain an EB
> greater than the previous consensus EB, a fork date is triggered at 6
> months using the smallest 5th percentile EB published. (Other times
> can be selected, but these are fairly conservative, looking for
> feedback here).Miner signalling is ignored during the waiting period.
>
> 4. Block heights used for timing
>
> 5. After 6 months, any users which already have the new EB or greater
> begin actually using it to validate transactions. Users use the EB or
> the latest 95% consensus triggered value - whichever is less.   This
> means that the portion of users that originally signaled for the
> increase do not have to upgrade their software to participate in the
> hard fork.
So anyone who does not change their EB are forked off of the network? If
the EB is an "advanced feature", then most users are going to be leaving
it at the default shipped with the software. That means that they will
then be forked off of the network when they don't change the EB because
it is an "advanced feature" that is more difficult to access.
>
> 6. Core can (optionally) ship a version with a default EB in-line with
> their own perceived consensus.  
>
> 7. Some sort of versioning system is used to ensure that the two
> networks (old and new) are incompatible... blocks hashed in one cannot
> be used in the other.
I think this would require a soft fork beforehand in order to implement
such a system.
>
> Any users which don't already have the new EB or greater should update
> their EB within the 6 month period - or they will be excluded from the
> majority fork.
>
> It would be in the best interests of major exchanges and users would
> to publicly announce their EB's.
Why?
>
> Users are free to safely set very high EB levels, based on their
> current hardware and network speeds. These EB levels don't cause those
> users to accept invalid blocks ever. They are safe because block size
> transitions behave like normal hard forks with high miner consensus (95%).
>
> No code changes will be needed to fork the network as many times as
> both users and miners feel the need to do so.  (Bitcoin core is off
> the hook for "scaling" issues...forever!)
"Scaling" includes a lot more than just the block size. There is much
more to scaling than just increasing the block size.
>
> If a smaller block size is needed, a reduced size can also be
> published and agreed upon by *both* users and miners using a the same
> mechanism, but the largest 5th percentile is used.   In other words...
> the requires broad consensus to deviate from status quo and fork.
>
> Any new node can simply follow these rules to validate all the blocks
> in a chain... even if the sizes changes a lot (at most twice per year).
What if the EB of a new node is set to be smaller than the current block
size?
>
>
>
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev