Re: [Lightning-dev] proposal for Lightning Network improvement proposals
> My concerns are mainly about assignment of values for types and feature > bits, and how we can test new features without conflicting with other > developers. There're enough feature bits that one can likely just pick a random high number, and start to roll out their experiment in the wild. If a collision _does_ happen, then your new feature/protocol will likely detect the incompatibility (example invalid message from your PoV), disconnect, and move on. Local feature bits of course, are much more lax than say, global feature bits. > For example, looking at the current BOLT#9, it appears bits 8/9 are the next > free bits available for use, There's no strict requirement that feature bits be assigned in order. They've been assigned in this order so far, as there really haven't been many new additions, or experiments in the wild. > however, I find that other developers have also opted to use the same bits: > https://github.com/rgb-org/spec/blob/master/04-lightning-network.md A simple courtesy to other developers would simply be to announce on the list that one is running an experiment with a new feature bit (bit 1493 or w/e) such that collisions in the wild are less likely. The authors of "RGB" to my knowledge have never posted on the list to "claim" their experimental feature bit. Also looking at the spec, their LN integration is more or less a hand wave over the actual implementations details, so really just a high level design sketch. -- Laolu On Mon, Jul 23, 2018 at 12:15 PM Mr Hodlen wrote: > Hi, new guy here. I've also been a bit confused about the process from > just > reading the BOLTs. > > I'm working on an implementation of the BOLTs with the intention of > experimenting with new features. > My concerns are mainly about assignment of values for types and feature > bits, > and how we can test new features without conflicting with other > developers. > > For example, looking at the current BOLT#9, it appears bits 8/9 are the > next > free bits available for use, so if I were implementing a new optional > feature, > I would pick the next available bits, > however, I find that other developers have also opted to use the same > bits: > https://github.com/rgb-org/spec/blob/master/04-lightning-network.md > > I think issue of requiring centralized number assignment can be easily > avoided > for experimental features by using a unique identifier for the feature > instead, > such as a hash of the feature name. > This should make it simpler for independent developers to try new features > before creating new pull requests for a BOLT assignment. > > Here's an extremely rough draft of the idea. The intent is that once an > experiment has been tested and is succesful, it can move towards being > standardized as a BOLT. Is it worth adding something like this? > > --- BOLT #(-1): Experimental Features [tentatively named] > > Overview > > The experimental features protocol is intended for testing protocol > features > which are not necessarily intended to become standard BOLTs. > > * Lightning base protocol messages: > > ** experiment > > A message for arbitrary protocol implementaion with a unique ID for each > feature supported. > > type: > data: > [32: experiment_name_hash] > [2: payloadlen] > [payloadlen: payload] > > `experiment_name_hash` is be a SHA256 of the feature name, in ASCII, > concatenated with a good source of randomness to avoid any conflict if any > two > developers chose the same name. > > `payloadlen` is the length of the remaining message, which MUST equal the > length of the outer message -34 bytes. > > `payload` is the payload of the experiment message, which should have the > same > format as a regular Lightning message described in BOLT #1. > > A sending node: > - MUST send the `init_experiments` message with `experiment_name_hash` > set > to 0 before any other `experiment` message is sent. > - MUST NOT send an experiment message with any non-zero > `experiment_name_hash` if the receiver has not already declared support > for > the feature_hash. > > A receiving node: > - If experiment_name_hash is zero: > - Treat the payload as the same format of lightning message, ie, a > 16- > bit big endian experiment_type followed by the data. > - If the experiment_name_hash is not recognized or disabled, MUST > ignore > the message. (SHOULD also fail the channels?) > > Rationale: > > The base lightning layer allows extension through feature bits and new odd > message types, but these require centralized assignment by the Lightning > community. > This creates a chicken and egg problem for developers testing experimental > features: In order to be assigned numbers, we need to standarize our > feature > through the BOLT process, > but we might not wish to attempt standardization until our experimental > features have been tested. > Without standardization, different teams working on different experimental > features may cause conflicts if different client
Re: [Lightning-dev] proposal for Lightning Network improvement proposals
Hi, new guy here. I've also been a bit confused about the process from just reading the BOLTs. I'm working on an implementation of the BOLTs with the intention of experimenting with new features. My concerns are mainly about assignment of values for types and feature bits, and how we can test new features without conflicting with other developers. For example, looking at the current BOLT#9, it appears bits 8/9 are the next free bits available for use, so if I were implementing a new optional feature, I would pick the next available bits, however, I find that other developers have also opted to use the same bits: https://github.com/rgb-org/spec/blob/master/04-lightning-network.md I think issue of requiring centralized number assignment can be easily avoided for experimental features by using a unique identifier for the feature instead, such as a hash of the feature name. This should make it simpler for independent developers to try new features before creating new pull requests for a BOLT assignment. Here's an extremely rough draft of the idea. The intent is that once an experiment has been tested and is succesful, it can move towards being standardized as a BOLT. Is it worth adding something like this? --- BOLT #(-1): Experimental Features [tentatively named] Overview The experimental features protocol is intended for testing protocol features which are not necessarily intended to become standard BOLTs. * Lightning base protocol messages: ** experiment A message for arbitrary protocol implementaion with a unique ID for each feature supported. type: data: [32: experiment_name_hash] [2: payloadlen] [payloadlen: payload] `experiment_name_hash` is be a SHA256 of the feature name, in ASCII, concatenated with a good source of randomness to avoid any conflict if any two developers chose the same name. `payloadlen` is the length of the remaining message, which MUST equal the length of the outer message -34 bytes. `payload` is the payload of the experiment message, which should have the same format as a regular Lightning message described in BOLT #1. A sending node: - MUST send the `init_experiments` message with `experiment_name_hash` set to 0 before any other `experiment` message is sent. - MUST NOT send an experiment message with any non-zero `experiment_name_hash` if the receiver has not already declared support for the feature_hash. A receiving node: - If experiment_name_hash is zero: - Treat the payload as the same format of lightning message, ie, a 16- bit big endian experiment_type followed by the data. - If the experiment_name_hash is not recognized or disabled, MUST ignore the message. (SHOULD also fail the channels?) Rationale: The base lightning layer allows extension through feature bits and new odd message types, but these require centralized assignment by the Lightning community. This creates a chicken and egg problem for developers testing experimental features: In order to be assigned numbers, we need to standarize our feature through the BOLT process, but we might not wish to attempt standardization until our experimental features have been tested. Without standardization, different teams working on different experimental features may cause conflicts if different clients are treating feature bits or message types differently. By moving all experimental features to a new message where they are wrapped in a unique feature name, this eradicates chance of conflicting implementations. Additionally, this message can serve as a generic transport mechanism between any two lightning nodes who have agreed to support the expierment_name_hash, as there is no restriction on the format of the payload. This may make it possible to serve e.g: HTTP over Lightning. * General experiment messages: If `experiment_name_hash` in the experiment message is 0, treat its payload as on of the following messages: ** init_experiments message Informs a peer of features supported by the client. 1. experiment_type: 16 2. data: * [2: eflen] * [eflen*32: experiment_name_hashes] A sending node: * MUST send the `init_experiments` message before any other `experiment` message for each connection. * SHOULD send the `experiment_name_hash` for any features supported and set to enabled in their software client. A receiving node: * For each experiment_name_hash: * If the hash is unknown or 0: Ignore the feature * If the hash is known: SHOULD enable the feature for communication with this peer. ** experiment_error message experiment_type: 17 data: [32: channel_id] [32: experiment_name_hash] [2: len] [len: data] For all messages before funding_created: Must use temporary_channel_id in lieu of channel_id. A sending node: * If error is critical, should also send the regular lightning `error` message from BOLT #1 * If the error is not specific to any channel: set channe
