Re: [bitcoin-dev] Packaged Transaction Relay
> Hi, > > Thanks for sharing your thoughts on packaged transaction relay. Hello, thanks for the reply. >> The sole objective, as expressed in the OP proposal, is to: >> "Propagate transactions that are incentive-compatible to mine, even >> if they don't meet minimum feerate alone." > > I actually do think there are additional goals we should include in any > protocol > change involving transaction relay, such as ensuring that we minimize > bandwidth waste as much as possible (as I mentioned in a previous message > in this thread). Yes - there is always the presumption of an optimally-performing protocol (not limited to bandwidth), this is just a restatement from the OP. The OP fails to eliminate orphan announcement, fails to prevent packages with insufficient fee from getting stuck in the same manner as txs (without explicitly re-announcing them again in an even larger package of higher feerate), and results in orphaned package announcements for the same reason (a static package is effectively just a larger tx). Due to the resulting orphaning, a node must allow its peer to continue to broadcast unverifiable orphans to it, potentially chasing ancestry. So in addition to bandwidth waste, there is also an inherent problem of bandwidth DOS. These are problems specifically addressed by packaged relay. [Regarding bandwidth waste: I've pointed out in years past that breaking the Bitcoin versioning scheme creates a requirement that any unknown message type be considered valid. Up until a recently-deployed protocol change, it had always been possible to validate messages by type. I noticed recently that validating nodes have been dropping peers at an increasing rate (a consequence of that deployment). Despite being an undocumented compatibility break, it is now unfortunately a matter of protocol that a peer must allow its peers to waste its bandwidth to remain compatible - something which we should eliminate.] > While I understand your proposal seeks to improve on an idea of static > packages in favor of dynamic package construction based on knowledge a > node should have of its peers, I think the main drawback of your proposal is > that it doesn't take into account the complexities of what a peer's "minimum > feerate" might mean. The consequence of this is that it's not generally > possible for a node to accurately guess whether a given transaction should > be sent in a package to a given peer, or not, and so in addition to any > "packaged transaction relay" mechanism that is implemented by a > transaction announcer, we'd still need to add protocol support for a receiving > peer to retrieve a package as well. It is certainly possible that there is ambiguity in BIP133 (and BIPs that modify it). However it's not clear to me which such ambiguity you are referring to. There is no guessing proposed. > First of all, a node's feerate is a dynamic value. BIP 133 allows for nodes > to > send feefilter messages at any time during the lifetime of a peer connection. > If we were to compare the feerate of ancestors of a relayed transaction to > the feerate in place at a peer as indicated by feefilter messages, and use > that > determine whether those ancestors would have been successfully relayed or > not, then doing so accurately would seem to require caching relay success for > each transaction, for each peer, at the time such transaction is relayed (or > perhaps caching feefilter values that are received from a peer?). This seems > likely to be undesirable, This is a possible implementation. What makes it undesirable? > and, at any rate, is more complex than merely comparing a pair of feerates. There are no necessary protocol changes (though a new INV type is ideal), so the relative complexity you are implying could only arise from implementation. While implementation considerations are relevant, achieving simplicity in the protocol is presumably the priority. Further, implementation complexity must be considered from what is necessary to actually achieve the objectives, and not from the perspective of any given implementation. Merely comparing a pair of feerates produces the problems described above, which includes not resolving the central problem, so this is an apples-to-oranges comparison. It's also more complex than doing nothing, but that also doesn't resolve the problem. > But even more fundamental than feerates being dynamic is that feerate > itself is not a well-defined concept when applied to arbitrary transaction > (sub-)graphs, and this is at the crux of the difficulty in coming up with > proposals that would meet the objective of ensuring that transactions which > are incentive-compatible to mine all get relayed successfully across the > network. Here are a couple examples that illustrate this: > > - Let A be a low feerate transaction (below any peer's minimum feerate). Let > B and C be descendants of A (but are otherwise unrelated). Suppose these >
Re: [bitcoin-dev] RFC for a BIP32 recurrent address derivation scheme
Hi Ruben, Thanks for your comments. I've noticed that there are lots of mentions of using a scheme like this, but there is no framework to ease the usage of such a scheme and to add interoperability between different implementations. So any implementation requires some manual work on both parties. The idea is to have a BIP to make this easy for developers to implement and users to use. The main advantage against silent payments or BIP47 is just that it should be easier to implement on both parties involved. Regarding the `contact`, you are right, it is just a counter, and Carol simply increments this one with each `contact` created. The association between a `contact` and the metadata of the contact needs to be stored off-chain, so when recovering the wallet that information is lost if there is no backup. Regarding the gap limit, I think that we can be quite strict with it, to make it easier to implement, I would use a gap limit of 2 for contacts and no gap limit for the index, there is no point in someone skipping an address. Regards. --- Eloy On Thu, Sep 29, 2022 at 7:41 PM Ruben Somsen wrote: > Hi Eloy, > > Nice idea. > > Note I thought about and succinctly described a similar scheme here (which > in turn was derived from work by Kixunil): > > https://gist.github.com/RubenSomsen/c43b79517e7cb701ebf77eec6dbb46b8#xpub-sharing > > I agree with your general assessment that this is a scheme that seems like > an improvement over the status quo. Note that both BIP47 and Silent > Payments don't require any interaction with the sender, while this scheme > requires one-time interaction (e.g. this wouldn't be suitable for one-time > donations). I think this would mostly be a convenience feature that > improves the regular interactive payment flow (interact once, instead of > repeatedly asking for addresses with each payment). > > >master / purpose' / coin_type' / contact' / index > > Despite your explanation, it's still not fully clear to me how "contact" > is defined, but I assume it's just a counter? Just in case, note that you > can't let Bob define it for Carol, as then you can't deterministically > recover your payments without also backing up how it's defined (the seed > alone won't be enough). > > The gap limit also needs to be kept in mind. If we allow each xpub to have > its own gap limit, you potentially get an exponential blowup (gaps in the > xpub * gaps in the addresses generated from the xpubs). It may be OK to > define a low default gap limit for these xpubs, since there should be no > reason to expect the same sender to leave any gaps, though this may depend > on how the xpubs are used (e.g. it may also be used to derive addresses for > others) so it's probably important to be explicit about this. > > Cheers, > Ruben > > > > On Thu, Sep 22, 2022 at 5:18 PM El_Hoy via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> There is a known issue on bitcoin, that is that every transaction >> requires a new address to prevent address reuse, making it uncomfortable to >> make recurring payments, as every payment requires a new off-chain >> interaction. A scheme is already mentioned on the [on the BIP32 itself][1], >> but it cannot be implemented as is. >> >> Here I propose a scheme that follows the structure described on [BIP44] >> that should make it possible to send recurring payments using a single >> offline interaction. >> >> The proposed scheme is: >> >> master / purpose' / coin_type' / contact' / index >> >> Where the definitions of all the levels follow BIP44, except for >> `contact` that is described below. >> >> Example usage: Bob wants to make recurring payments to Carol, so he asks >> her for a _contact address_, that is, an extended public key. >> >> Bob can use that public key to generate multiple derived addresses to >> make multiple recurring payments to Carol, the contact address is stored >> off-chain, anyone inspecting the chain will just see normal transactions >> on-chain. >> >> ## Considerations >> >> [BIP47] tries to solve the same issue, but the solution is more complex >> and involves more on-chain transactions that involve data, this >> implementation simpler and requires less work to implement. >> >> Also, the derivation path might need some adjustments for different >> address types on bitcoin. >> >> Finally, this only works in a single direction and does not make it >> possible for Carol to send anything to Bob, as it would require Bob sending >> her a contact address. >> >> ## Advantages >> >> A positive side effect of using this, is that Bob can choose to send >> payments to Carol using multiple outputs, giving him more privacy. >> >> Also, those payments can be easily labeled by the receiving wallet, as >> they are received. >> >> Regards. >> >> ### References >> >> [1]: >> https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki#recurrent-business-to-business-transactions-nmih0 >> [BIP47]:
Re: [bitcoin-dev] Packaged Transaction Relay
(Apologies for the double-post -- I'm resending this message to the list with much of the quoted text trimmed, because my first message was placed in the moderation queue for being too large) Hi, Thanks for sharing your thoughts on packaged transaction relay. The sole objective, as expressed in the OP proposal, is to: "Propagate transactions that are incentive-compatible to mine, even if they > don't meet minimum feerate alone." I actually do think there are additional goals we should include in any protocol change involving transaction relay, such as ensuring that we minimize bandwidth waste as much as possible (as I mentioned in a previous message in this thread). While I understand your proposal seeks to improve on an idea of static packages in favor of dynamic package construction based on knowledge a node should have of its peers, I think the main drawback of your proposal is that it doesn't take into account the complexities of what a peer's "minimum feerate" might mean. The consequence of this is that it's not generally possible for a node to accurately guess whether a given transaction should be sent in a package to a given peer, or not, and so in addition to any "packaged transaction relay" mechanism that is implemented by a transaction announcer, we'd still need to add protocol support for a receiving peer to retrieve a package as well. First of all, a node's feerate is a dynamic value. BIP 133 allows for nodes to send feefilter messages at any time during the lifetime of a peer connection. If we were to compare the feerate of ancestors of a relayed transaction to the feerate in place at a peer as indicated by feefilter messages, and use that determine whether those ancestors would have been successfully relayed or not, then doing so accurately would seem to require caching relay success for each transaction, for each peer, at the time such transaction is relayed (or perhaps caching feefilter values that are received from a peer?). This seems likely to be undesirable, and, at any rate, is more complex than merely comparing a pair of feerates. But even more fundamental than feerates being dynamic is that feerate itself is not a well-defined concept when applied to arbitrary transaction (sub-)graphs, and this is at the crux of the difficulty in coming up with proposals that would meet the objective of ensuring that transactions which are incentive-compatible to mine all get relayed successfully across the network. Here are a couple examples that illustrate this: - Let A be a low feerate transaction (below any peer's minimum feerate). Let B and C be descendants of A (but are otherwise unrelated). Suppose these transactions are relayed to a node in the order A, B, C. In the algorithm you proposed, I believe the determination for whether C should be announced to a given peer as a package (A, C) or as a singleton would either (1) depend on whether the package (A, B) was sufficient to meet the peer's feerate, or (2) waste bandwidth by engaging in packaged relay whenever A was already successfully relayed as part of a package. Both of these approaches seem undesirable. - Let A be a high fee, but low feerate transaction. Suppose B is a transaction that conflicts with A, has a high feerate, but lower total fee. In this situation, two different nodes that learned of these two transactions in opposite order [(A, B) vs (B, A)] might be expected to have differing mempools -- this at least would be the case in the BIP 125 algorithm (which requires that both feerate and total fee must increase when replacing an existing transaction), and at any rate it's not obvious from the information given which would be more optimal to include in a block, as that depends on factors that go beyond just these two transactions. Suppose further that a new transaction C is relayed on the network, which is a child of B and very high feerate, such that B + C have higher fee and higher feerate than A, when taken together. In this case we'd want our relay protocol to ensure that even nodes which saw A first should still have a chance to consider transaction C, but the packaging design you propose (which would compare transaction B's feerate to the peer's, and conclude packaging is unnecessary because B's feerate might already exceed the peer's feerate) would not facilitate this. To summarize, the point I'd make from these examples is that we should not expect that "feerate" (whatever that means) alone will be a sufficient predictor of what is in our peer's mempools. So while there may be some situations where a transaction relayer might be able to usefully package up a transaction with its dependencies (perhaps in narrowly defined situations), there will also always be situations where this isn't possible, and what I conclude from that is that it should be helpful to add to the protocol some way for the recipient of a transaction to request the dependencies directly. Taken together, I roughly understand
