Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility
В Thu, 01 Aug 2019 19:01:06 +
Andrew Chow wrote:
> I spoke to some people OOB and they said that they didn't really like
> the idea of having a prefix string (partially because they've already
> implemented some proprietary types by simply squatting on unused
> types). Matching the prefix string adds additional complexity to the
> parser code.
I do not oppose the idea of "{0xFC}|{private_type}" strongly, but I
would like to note that for those who do not want to deal with
additional complexity of handling a prefixed string, they can simply
not use it at all. Since this is a private construction, and their
private format specifies 'no prefix', they can just ignore everything
that does not start with "{0xFC}|{0x00}", thus any further complexity
regarding the prefix is also ignored. The only added complexity is one
condition check: second_byte_of_the_key != 0
My other argument for conflict-avoidance prefix as a first thing after
0xFC is that the set of future users of PSBT and private types is
most likely much larger than the current set of those who already
implemented proprietary types on their own, and thus the overall benefit
for the whole industry will likely be bigger when 'i do not want
conflict avoidance' decision have to be explicit, by setting the prefix
to 0x00, and the set of possible conflicting types are limited only to
those entities that made this explicit decision.
Regarding the 'squatted' types, it seems to me that this only matters
in the discussed context if they squatted on 0xFC type in particular.
In other cases, they will need to implement changes anyway, to be
compatible with the BIP. Maybe they could consider that one additional
condition check is a small burden, and maybe they can tolerate that,
for the benefit of reducing possibility of interoperability problems
between other future PSBT/private types implementors.
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility
I spoke to some people OOB and they said that they didn't really like
the idea of having a prefix string (partially because they've already
implemented some proprietary types by simply squatting on unused types).
Matching the prefix string adds additional complexity to the parser
code. The main concern is that people won't want to actually follow the
spec for proprietary types and instead just use some unused type value.
So I think instead we should do:
{0xFC}|{private_type}
and the prefix string can be optional (and strongly recommended) after that.
Since public parsers won't really be enforcing the rules for proprietary
types, I don't think it really makes sense to specify and enforce how
they should be. After all, the key is really just an opaque data blob.
In the same vein, it would probably be useful to allow multiple types
for proprietary use as originally proposed to make implementation of
these easier. If more type are needed, then the private type
construction can be used.
Andrew
On 8/1/19 1:57 PM, Andrew Chow wrote:
>
> It seems like the consensus is that we should use Compact Size Unsigned
> Integers for the field types, so we will do that. The types will be
> minimally encoded CSUints.
>
> For the proprietary types, I will use Dimitry's and Andrew Poelstra's
> (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-March/016713.html)
> suggestion. There will be one proprietary type, 0xFC. This will be
> followed by a variable length string that is a vendor specific prefix
> that serves as a unique identifier to help with preventing usage of
> proprietary types outside of private contexts. This will then be
> followed by the actual type for the data, as defined by the user of the
> proprietary type.
>
> The prefix will just be a string, prefixed with a compact size unsigned
> integer. If no prefix is wanted, then a single 0x00 byte can be used.
>
> For public software, there is no need to handle these proprietary types
> at all so they do not need to check the string or the data type. It is
> not necessary to enforce the above rules about proprietary types except
> that they use the proprietary type value.
>
>
> Andrew Chow
>
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility
It seems like the consensus is that we should use Compact Size Unsigned Integers for the field types, so we will do that. The types will be minimally encoded CSUints. For the proprietary types, I will use Dimitry's and Andrew Poelstra's (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-March/016713.html) suggestion. There will be one proprietary type, 0xFC. This will be followed by a variable length string that is a vendor specific prefix that serves as a unique identifier to help with preventing usage of proprietary types outside of private contexts. This will then be followed by the actual type for the data, as defined by the user of the proprietary type. The prefix will just be a string, prefixed with a compact size unsigned integer. If no prefix is wanted, then a single 0x00 byte can be used. For public software, there is no need to handle these proprietary types at all so they do not need to check the string or the data type. It is not necessary to enforce the above rules about proprietary types except that they use the proprietary type value. Andrew Chow ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility
> why not use Bitcoin compact uint, which most PSBT consumers already implement? I totally agree with that, compact int parsing is already implemented in all bitcoin applications, wallets and libraries. Also, for certain (mb proprietary) applications I will be willing to use multi-byte keys where the first byte defines the application type and next bytes define application-specific fields. I would suggest to set proprietary bytes to 0xF0-0xFC or to 0xE0-0xEF then (E for Enterprise?). ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility
Hi, On 7/31/19 12:19 PM, Dmitry Petukhov wrote: > > I think private formats should have at least a basic format: they > should start with a prefix. This way different prviate formats can be > distinguished by this prefix, and there will be no risk of > unintentional confusion. > > Private types can start with the size of the prefix, and then > organization can choose any prefix they like, or no prefix, if > the size is of the prefix is 0 (means they are fine with possible > conflicts with other empty-prefix private types) > I don't think that should something that is required for people to do, but perhaps it can be something that is strongly recommended and suggested in the BIP itself. > > Why not just say that the types should be encoded as 'compact size > unsigned integer' ? This format for variable length integer encoding is > already used in the BIP for other fields, and thus will not add any > additional complexity to the parsing. > On 7/31/19 10:32 AM, jan matejek via bitcoin-dev wrote:> > > why not use Bitcoin compact uint, which most PSBT consumers already > implement? > There are a few issues with using a compact size uint. The main issue is that it doesn't translate well to the proprietary use types. If we used CSUint for the type, then all of type values for proprietary use need to be reserved instead of allowing them to be infinitely expanded from the initial set of proprietary use types. There is also the fact that CSUints are malleable as the same value can be represented in many different ways, just with different amounts of leading zeroes. But I suppose that isn't really that big of an issue. I am not opposed to using a CSUint, I just felt that it made things a bit harder and was unnecessary. Andrew ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility
В Wed, 31 Jul 2019 01:13:46 + Andrew Chow via bitcoin-dev wrote: > Firstly, I would like to propose that some types be reserved for > proprietary use. These proprietary use types are, in general, for > private use by individuals and organizations who want to use PSBT in > their processes. These are usefule when there are additional data that > they need attached to a PSBT but such data are not useful (or > available) for the general public. I think private formats should have at least a basic format: they should start with a prefix. This way different prviate formats can be distinguished by this prefix, and there will be no risk of unintentional confusion. Private types can start with the size of the prefix, and then organization can choose any prefix they like, or no prefix, if the size is of the prefix is 0 (means they are fine with possible conflicts with other empty-prefix private types) > Lastly, I would like to propose the canonical method for mult-byte > types. We designate a specific type to indicate that the type is > multiple bytes. When such types are observed, parsers should move onto > the next byte and interpret that as the type, keeping in mind the > number of bytes that were read in for the type. > > I propose that we use 0xFF as this designated type. When a parser sees > an 0xFF value as the type, it reads the next byte as being part of the > type. So two byte types will be of the form 0xFFXX. This method allows > us to do a prefix match in order to quickly identify the type being > used. For types with more bytes, simply use another 0xFF byte. So > three byte types would be of the form 0xXX, four byte, > 0xFFXX, and so on. When multi-byte types are specified in the > BIP, they should be specified in this full length form, i.e. two byte > types as 0xFFXX. Why not just say that the types should be encoded as 'compact size unsigned integer' ? This format for variable length integer encoding is already used in the BIP for other fields, and thus will not add any additional complexity to the parsing. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility
hello, On 31. 07. 19 3:13, Andrew Chow via bitcoin-dev wrote: > Lastly, I would like to propose the canonical method for mult-byte > types. We designate a specific type to indicate that the type is > multiple bytes. When such types are observed, parsers should move onto > the next byte and interpret that as the type, keeping in mind the number > of bytes that were read in for the type. why not use Bitcoin compact uint, which most PSBT consumers already implement? regards m. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[bitcoin-dev] Proposed Extensions to BIP 174 for Future Extensibility
Hi All, I would like to propose some types that allow for BIP 174 PSBT to be extended more in the future. Firstly, I would like to propose that some types be reserved for proprietary use. These proprietary use types are, in general, for private use by individuals and organizations who want to use PSBT in their processes. These are usefule when there are additional data that they need attached to a PSBT but such data are not useful (or available) for the general public. These types will be guaranteed to not be used by the public specification and there is no expectation that any publicly available software be able to understand any specific meanings of these types. These types should be used for internal processes only. The types I would like to reserve for proprietary use are the 15 types from 0xF0 to 0xFE inclusive. These 15 type values will be the same for global, per-input, and per-output types. If 15 types are not enough, additional types can be obtained using the multi-byte type method described later. Next, I would like to propose a global version type and field. The version type is 0xEF with only the type as the key, and the value is a 32-bit unsigned little endian integer representing the version number. A PSBT without a version number is to be considered version 0. If a parser sees a version number that it does not understand, it should exit immediately as the PSBT will contain types that are not safe to ignore. This version number is a safeguard in the event that a backwards incompatible change is introduce to PSBT. While PSBT is designed and intended to be forwards compatible by allowing parsers to ignore types that they do not understand, it is possible that at ype is added in the future which breaks this assumption and it would be unsafe for a type to be ignored. Updaters and combiners that need to add a version number to a PSBT should use the highest version number required. For example, if a combiner sees two PSBTs for the same transaction, one with version 0, and the other with version 1, then it should combine them and produce a PSBT with version 1. If an updater is updating a PSBT and needs to add a field that is only available in version 1, then it should set the PSBT version number to 1 unless a version higher than that is already specified. It is not expected that the version number will ever be used. We try to make PSBT fields safe to ignore. The version number is only being included here as a safeguard in the event that breaking compatibilty is required. Lastly, I would like to propose the canonical method for mult-byte types. We designate a specific type to indicate that the type is multiple bytes. When such types are observed, parsers should move onto the next byte and interpret that as the type, keeping in mind the number of bytes that were read in for the type. I propose that we use 0xFF as this designated type. When a parser sees an 0xFF value as the type, it reads the next byte as being part of the type. So two byte types will be of the form 0xFFXX. This method allows us to do a prefix match in order to quickly identify the type being used. For types with more bytes, simply use another 0xFF byte. So three byte types would be of the form 0xXX, four byte, 0xFFXX, and so on. When multi-byte types are specified in the BIP, they should be specified in this full length form, i.e. two byte types as 0xFFXX. The same mechanism can be used for the proprietary use types, just with a different value as the designated multi-byte indicator. For example, one could use 0xFE as the designated type as that is in the proprietary types range. Of course any type within the proprietary type range could be used as the indicator, it is up to the users to determine this themselves. While other methods of indicating multiple bytes and lengths may be more space efficient and allow us to have more types represented in a smaller space, I am choosing this method because of its simplicity. This is easy to understand and implement. Furthermore, I do not expect that we will use so many types. I don't think that we will need to have more than one byte types for a very long time. Please let me know your thoughts on these extensions. I will open a PR to the bips repo to add these to BIP 174 if there are no objections. Andrew Chow ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
