I should not take it that the lack of critical feedback to this revised proposal is a glowing endorsement. I understand that there would be technical issues to resolve in implementation, but, are there no fundamental errors?
I suppose that it if is difficult to determine how long a transaction has been waiting in the pool then, each node could simply keep track of when a transaction was first seen. This may have implications for a verify routine, however, for example, if a node was offline, how should it differentiate how long each transaction was waiting in that case? If a node was restarted daily would it always think that all transactions had been waiting in the pool less than one day If each node keeps the current transaction pool in a file and updates it, as transactions are included in blocks and, as new transactions appear in the pool, then that would go some way to alleviate the issue, apart from entirely new nodes. There should be no reason the contents of a transaction pool files cannot be shared without agreement as to the transaction pool between nodes, just as nodes transmit new transactions freely. It has been questioned why miners could not cheat. For the question of how many transactions to include in a block, I say it is a standoff and miners will conform to the proposal, not wanting to leave transactions with valid fees standing, and, not wanting to shrink the transaction pool. In any case, if miners shrink the transaction pool then I am not immediately concerned since it provides a more efficient service. For the question of including transactions according to the proposal, I say if it is possible to keep track of how long transactions are waiting in the pool so that they can be included on a probability curve then it is possible to verify that blocks conform to the proposal, since the input is a probability, the output should conform to a probability curve. If someone has the necessary skill, would anyone be willing to develop the math necessary for the proposal? Regards, Damian Williamson ________________________________ From: bitcoin-dev-boun...@lists.linuxfoundation.org <bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> Sent: Friday, 8 December 2017 8:01 AM To: bitcoin-dev@lists.linuxfoundation.org Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks Good afternoon, The need for this proposal: We all must learn to admit that transaction bandwidth is still lurking as a serious issue for the operation, reliability, safety, consumer acceptance, uptake and, for the value of Bitcoin. I recently sent a payment which was not urgent so; I chose three-day target confirmation from the fee recommendation. That transaction has still not confirmed after now more than six days - even waiting twice as long seems quite reasonable to me. That transaction is a valid transaction; it is not rubbish, junk or, spam. Under the current model with transaction bandwidth limitation, the longer a transaction waits, the less likely it is ever to confirm due to rising transaction numbers and being pushed back by transactions with rising fees. I argue that no transactions are rubbish or junk, only some zero fee transactions might be spam. Having an ever-increasing number of valid transactions that do not confirm as more new transactions with higher fees are created is the opposite of operating a robust, reliable transaction system. Business cannot operate with a model where transactions may or may not confirm. Even a business choosing a modest fee has no guarantee that their valid transaction will not be shuffled down by new transactions to the realm of never confirming after it is created. Consumers also will not accept this model as Bitcoin expands. If Bitcoin cannot be a reliable payment system for confirmed transactions then consumers, by and large, will simply not accept the model once they understand. Bitcoin will be a dirty payment system, and this will kill the value of Bitcoin. Under the current system, a minority of transactions will eventually be the lucky few who have fees high enough to escape being pushed down the list. Once there are more than x transactions (transaction bandwidth limit) every ten minutes, only those choosing twenty-minute confirmation (2 blocks) will have initially at most a fifty percent chance of ever having their payment confirm. Presently, not even using fee recommendations can ensure a sufficiently high fee is paid to ensure transaction confirmation. I also argue that the current auction model for limited transaction bandwidth is wrong, is not suitable for a reliable transaction system and, is wrong for Bitcoin. All transactions must confirm in due time. Currently, Bitcoin is not a safe way to send payments. I do not believe that consumers and business are against paying fees, even high fees. What is required is operational reliability. This great issue needs to be resolved for the safety and reliability of Bitcoin. The time to resolve issues in commerce is before they become great big issues. The time to resolve this issue is now. We must have the foresight to identify and resolve problems before they trip us over. Simply doubling block sizes every so often is reactionary and is not a reliable permanent solution. I have written a BIP proposal for a technical solution but, need your help to write it up to an acceptable standard to be a full BIP. I have formatted the following with markdown which is human readable so, I hope nobody minds. I have done as much with this proposal as I feel that I am able so far but continue to take your feedback. # BIP Proposal: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks ## The problem: Everybody wants value. Miners want to maximize revenue from fees (and we presume, to minimize block size). Consumers need transaction reliability and, (we presume) want low fees. The current transaction bandwidth limit is a limiting factor for both. As the operational safety of transactions is limited, so is consumer confidence as they realize the issue and, accordingly, uptake is limited. Fees are artificially inflated due to bandwidth limitations while failing to provide a full confirmation service for all transactions. Current fee recommendations provide no satisfaction for transaction reliability and, as Bitcoin scales, this will worsen. Bitcoin must be a fully scalable and reliable service, providing full transaction confirmation for every valid transaction. The possibility to send a transaction with a fee lower than one that is acceptable to allow eventual transaction confirmation should be removed from the protocol and also from the user interface. ## Solution summary: Provide each transaction with an individual transaction priority each time before choosing transactions to include in the current block, the priority being a function of the fee paid (on a curve), and the time waiting in the transaction pool (also on a curve) out to n days (n=60 ?). The transaction priority to serve as the likelihood of a transaction being included in the current block, and for determining the order in which transactions are tried to see if they will be included. Use a target block size. Determine the target block size using; current transaction pool size x ( 1 / (144 x n days ) ) = number of transactions to be included in the current block. Broadcast the next target block size with the current block when it is solved so that nodes know the next target block size for the block that they are building on. The curves used for the priority of transactions would have to be appropriate. Perhaps a mathematician with experience in probability can develop the right formulae. My thinking is a steep curve. I suppose that the probability of all transactions should probably account for a sufficient number of inclusions that the target block size is met although, it may not always be. As a suggestion, consider including some zero fee transactions to pad, highest BTC value first? **Explanation of the operation of priority:** > If transaction priority is, for example, a number between one (low) and > one-hundred (high) it can be directly understood as the percentage chance in > one-hundred of a transaction being included in the block. Using probability > or likelihood infers that there is some function of random. If random (100) < > transaction priority then the transaction is included. >To break it down further, if both the fee on a curve value and the time >waiting on a curve value are each a number between one and one-hundred, a >rudimentary method may be to simply multiply those two numbers, to find the >priority number. For example, a middle fee transaction waiting thirty days (if >n = 60 days) may have a value of five for each part (yes, just five, the >values are on a curve). When multiplied that will give a priority value of >twenty-five, or, a twenty-five percent chance at that moment of being >included in the block; it will likely be included in one of the next four >blocks, getting more likely each chance. If it is still not included then the >value of time waiting will be higher, making for more probability. A very low >fee transaction would have a value for the fee of one. It would not be until >near sixty-days that the particular low fee transaction has a high likelihood >of being included in the block. I am not concerned with low (or high) transaction fees, the primary reason for addressing the issue is to ensure transactional reliability and scalability while having each transaction confirm in due time. ## Pros: * Maximizes transaction reliability. * Fully scalable. * Maximizes possibility for consumer and business uptake. * Maximizes total fees paid per block without reducing reliability; because of reliability, in time confidence and overall uptake are greater; therefore, more transactions. * Market determines fee paid for transaction priority. * Fee recommendations work all the way out to 30 days or greater. * Provides additional block entropy; greater security since there is less probability of predicting the next block. ## Cons: * Could initially lower total transaction fees per block. * Must be first be programmed. ## Solution operation: This is a simplistic view of the operation. The actual operation will need to be determined in a spec for the programmer. 1. Determine the target block size for the current block. 2. Assign a transaction priority to each transaction in the pool. 3. Select transactions to include in the current block using probability in transaction priority order until the target block size is met. 5. Solve block. 6. Broadcast the next target block size with the current block when it is solved. 7. Block is received. 8. Block verification process. 9. Accept/reject block based on verification result. 10. Repeat. ## Closing comments: It may be possible to verify blocks conform to the proposal by showing that the probability for all transactions included in the block statistically conforms to a probability distribution curve, *if* the individual transaction priority can be recreated. I am not that deep into the mathematics; however, it may also be possible to use a similar method to do this just based on the fee, that statistically, the blocks conform to a fee distribution. Any zero fee transactions would have to be ignored. This solution needs a clever mathematician. I implore, at the very least, that we use some method that validates full transaction reliability and enables scalability of block sizes. If not this proposal, an alternative. Regards, Damian Williamson
_______________________________________________ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
