Re: [bitcoin-dev] [bitcoin-discuss] Proposal to replace full blockchain with recent history plus UTXO Set

[Damian Williamson via bitcoin-dev]

A fairly decent rework would be needed but it seems that the idea has merit 
initially.


As it is now, it is not only that a utxo exists but, that the transaction it 
references and the block it is within can also be fully validated.


So, if a utxo block set type existed then by consensus every so often a bunch 
of blocks containing just the validated utxo set to a given height, say 100,000 
blocks below the current blockheight, and necessary header data could be 
appended onto the valid chain and nodes would be free to drop all preceding 
blocks. I suspect that many wouldn't and that even many new nodes would still 
desire to download the full blockchain but, for the use case you mention it 
would make sense.


If done [right/wrong] it may even make Satoshi's fortune spendable. Something 
to watch out for.


From: bitcoin-discuss-boun...@lists.linuxfoundation.org 
 on behalf of Dave Scotese 
via bitcoin-discuss 
Sent: Wednesday, 26 September 2018 1:46:54 AM
To: Bitcoin Discuss
Subject: Re: [bitcoin-discuss] Proposal to replace full blockchain with recent 
history plus UTXO Set

The image at imgur and the pastebin both reference block 542324 but the correct 
block is 542322.  As the pastebin shows, the decimal and hex representations I 
gave for the block height did not match, and this is why.  If you use the 
Merkle root for block 542322 instead of 542324, you'll be able to see the 
correct Game of Life play out and make the apron image.

Dave.

On Sun, Sep 23, 2018 at 11:38 AM Dave Scotese 
mailto:dscot...@litmocracy.com>> wrote:
I thought I didn't have access to the dev list and so intended to post the 
following proposal to this discussion list, but used the wrong email address.  
Anyway, my email did get into the dev list 
(https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-September/016420.html)
 but I'll repeat it here:

I've been working on an idea that relieves full nodes of storing the entire 
blockchain. Open source software generally relies on the fact that "enough" 
people agree that it's secure. Bitcoin software works that way too. So if you 
understand enough to see that a UTXO set is valid at a certain block height, 
and there are enough other people who agree and that set is recognizable by 
humans, then we can use that UTXO set and ditch the blockchain that existed up 
to that point. It would save a lot of storage and make it a lot easier to run a 
full node.

Have you reviewed the source code from which your wallets were compiled? At 
some point, we all trust third parties, but generally (at least among people 
who understand Bitcoin) they are large composite groups so that no small group 
or individual can profit from cheating.

I look forward to answering any concerns and also to any offers of help.   I 
used block 542324 of the Bitcoin blockchain to make a memorable experience 
using the game of life. I wrote a script for the open-source Game-of-Life 
software Golly and shared it in the paste at https://pastebin.com/k5Ssc0qk. It 
produces the image at https://imgur.com/a/rwIQuVz. If someone can tell me how 
to get a UTXO Set from the bitcoin client, I'll send them $50 of bitcoin. Then 
I could get the SHA256 hash of that set and try to make a recognizable 
checkpoint for the Bitcoin blockchain. If someone runs Golly and shares a video 
of the game playing out (into the apron-shaped image), I'll send them $50 of 
bitcoin too.

In a few decades when the blockchain has grown to a few terabytes and the UTXO 
Set is still just a few gigabytes, I'd like to see more people start running 
full nodes without the hassle of a long wait and loads of storage space. That's 
what stops me from running one.


--
I like to provide some work at no charge to prove my value. Do you need a 
techie?
I own Litmocracy and Meme 
Racing (in alpha).
I'm the webmaster for The Voluntaryist which now 
accepts Bitcoin.
I also code for The Dollar Vigilante.
"He ought to find it more profitable to play by the rules" - Satoshi Nakamoto
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Selfish Mining Prevention

[Damian Williamson via bitcoin-dev]

I see what you say, however, since the proposal as I have read it says "And 
this will keep happening as long as hashrate keeps rising," - what actually 
happens in the case hashrate stagnates or falls?


I would prefer to see (not only with your proposal) greater bias toward 
hashrate being exponentially more uneconomical the more it rises to stifle 
greed. The current level of mining already greatly exceeds that necessary for 
the stability of the network and far lower hashrates are completely acceptible 
provided the network is protected from large switch-ons, which I say is 
achievable.


I do have other thoughts to approach greed that I have not yet made formal on 
this list, much unrelated to your proposal, but, I see freedom of use of 
Bitcoin needing to be censorship resistant but not necessarily mining provided 
it is protected enough or free or flexible enough to allow for, say, 50k 
globally distributed standard mining hardware units to exist operating at any 
one time profitably. That said, I am PoW only and not PoS orientated.


From: akarama...@gmail.com  on behalf of Andrew 

Sent: Sunday, 16 September 2018 2:01:19 AM
To: Damian Williamson
Cc: Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] Selfish Mining Prevention

@Moral Agent: No problem. I did ask in the first post what the current
plans are for selfish miner prevention. So if anyone has any other
relevant ideas (not just for selfish mining but for making mining more
decentralized and competetive), then please post it, but I just prefer
to focus on my proposal more than others.

@Damian: I think you are concerned that this will incentivize more
global resource consumption and will be detrimental to our
environment? Personally, I believe centralization of energy does more
harm to the environment rather than total energy consumption. If
Bitcoin helps "power" to become more decentralized, then I wouldn't be
surprised if total (global) energy consumption actually decreases. The
debt based economy is forcing us to continuously grow and use up more
resources, and collectivism is turning individuals into
super-organisms capable of doing much more harm to the environment
than can be done by one or a just a few individuals working
independently. In my proposal, I actually allow for changing
environmental conditions by measuring only the peak hashrate of the
past year, and not the full history of blocks.

On Sat, Sep 15, 2018 at 5:29 AM, Damian Williamson  wrote:
>>This "reserve" part of the fee will be paid to miners if the hashrate
>> rises.
>
>
> Anticipating ongoing hashrate rise shows that you have not yet thought about
> moving outside of the current greed model, a model wherein mining will
> consume all available resources within the colony's objective just to spread
> as far as possible with each new miner bringing diminishing individual
> returns and shortening the life of Earth for no additional gain. Greed model
> := bacteria.
>
> 
> From: bitcoin-dev-boun...@lists.linuxfoundation.org
>  on behalf of Andrew via
> bitcoin-dev 
> Sent: Friday, 14 September 2018 9:19:37 AM
> To: Bitcoin Dev
> Subject: Re: [bitcoin-dev] Selfish Mining Prevention
>
> I discussed this more at bitcointalk:
> https://bitcointalk.org/index.php?topic=4998410.0
>
> The attacks I'm interested in preventing are not only selfish mining
> and collusion, but also more subtle attacks like block withholding,
> and in general anything that aims to drive out the competition in
> order to increase hashrate fraction. I also scrapped the idea of
> changing the block subsidies, and I am only focuses on fees.
>
> You can read more about the motivation and details in the bitcointalk
> thread, but my proposal in short would be to add the concept of
> "reserve fees". When a user makes a transaction, for each txout
> script, they can add parameters that specify the fraction of the total
> fee that is held in "reserve" and the time it is held in "reserve"
> (can set a limit of 2016 blocks). This "reserve" part of the fee will
> be paid to miners if the hashrate rises. So if hashrate is currently h
> and peak hashrate (from past year) is p, then for each period (1 day),
> a new hashrate is calculated h1, and if h1 > h, then the fraction
> (h1-h)/p from the reserve fees created in the past 2016 blocks will be
> released to miners for that period (spread out over the 144 blocks in
> that period). And this will keep happening as long as hashrate keeps
> rising, until the "contract" expires, and the leftover part can be
> used by the owner of the unspent output, but it can only be used for
> paying fees, not as inputs for future transactions (to save on block
> space).
>
> This should incentivize miners to not drive out the competition, since
> if they do, there will be less of these reserve fees given to miners.
> Yes in the end the miners will get all the fees, but with rising
> hashrate they get an unconditional 

Re: [bitcoin-dev] Selfish Mining Prevention

[Damian Williamson via bitcoin-dev]

>This "reserve" part of the fee will be paid to miners if the hashrate rises.


Anticipating ongoing hashrate rise shows that you have not yet thought about 
moving outside of the current greed model, a model wherein mining will consume 
all available resources within the colony's objective just to spread as far as 
possible with each new miner bringing diminishing individual returns and 
shortening the life of Earth for no additional gain. Greed model := bacteria.


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
 on behalf of Andrew via 
bitcoin-dev 
Sent: Friday, 14 September 2018 9:19:37 AM
To: Bitcoin Dev
Subject: Re: [bitcoin-dev] Selfish Mining Prevention

I discussed this more at bitcointalk:
https://bitcointalk.org/index.php?topic=4998410.0

The attacks I'm interested in preventing are not only selfish mining
and collusion, but also more subtle attacks like block withholding,
and in general anything that aims to drive out the competition in
order to increase hashrate fraction. I also scrapped the idea of
changing the block subsidies, and I am only focuses on fees.

You can read more about the motivation and details in the bitcointalk
thread, but my proposal in short would be to add the concept of
"reserve fees". When a user makes a transaction, for each txout
script, they can add parameters that specify the fraction of the total
fee that is held in "reserve" and the time it is held in "reserve"
(can set a limit of 2016 blocks). This "reserve" part of the fee will
be paid to miners if the hashrate rises. So if hashrate is currently h
and peak hashrate (from past year) is p, then for each period (1 day),
a new hashrate is calculated h1, and if h1 > h, then the fraction
(h1-h)/p from the reserve fees created in the past 2016 blocks will be
released to miners for that period (spread out over the 144 blocks in
that period). And this will keep happening as long as hashrate keeps
rising, until the "contract" expires, and the leftover part can be
used by the owner of the unspent output, but it can only be used for
paying fees, not as inputs for future transactions (to save on block
space).

This should incentivize miners to not drive out the competition, since
if they do, there will be less of these reserve fees given to miners.
Yes in the end the miners will get all the fees, but with rising
hashrate they get an unconditional subsidy that does not require
transactions, thus more space for transactions with fees.

I can make a formal BIP and pull request, but I need to know if there
is interest in this. Now fees don't play such a large part of the
block reward, but they will get more important, and this change
wouldn't force anything (would be voluntary by each user), just miners
have to agree to it with a soft fork (so they don't spend from the
anyone-can-spend outputs used for reserve fees). Resource requirements
for validation are quite small I believe.

On Sat, Sep 1, 2018 at 12:11 AM, Andrew  wrote:
> As I understand, selfish mining is an attack where miners collude to
> mine at a lower hashrate then with all miners working independently.
> What are the current strategies used to prevent this and what are the
> future plans?
>
> One idea I have is to let the block reward get "modulated" according
> to peak hashrate. Say p is the peak hashrate for 365 periods (1 year)
> consisting of 144 blocks, h is the hashrate of the last 144 block (1
> day) period, and r is the base subsidy (12.5 BTC currently). You can
> then make the max block reward 0.5 r (1 + h/p). So if hashrate is at
> peak you get the full reward. Otherwise you get less, down to a min of
> 0.5 r.
>
> If miners were to collude to mine at a lower than peak hashrate, then
> they may be able to do it profitably for 144 blocks, but after that,
> the reward would get modulated and it wouldn't be so much in their
> interest to continue mining at the lower hashrate.
>
> What flaws are there with this? I know it could be controversial due
> to easier mining present for early miners, so maybe it would have to
> be done in combination with a new more dynamic difficulty adjustment
> algorithm. But I don't see how hashrate can continue rising
> indefinitely, so a solution should be made for selfish mining.
>
> Also when subsidies stop and a fee market is needed, I guess a portion
> of the fees can be withheld for later if hashrate is not at peak.
>
>
> --
> PGP: B6AC 822C 451D 6304 6A28  49E9 7DB7 011C D53B 5647



--
PGP: B6AC 822C 451D 6304 6A28  49E9 7DB7 011C D53B 5647
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Guiding transaction fees towards a more censorship resistant outcome

[Damian Williamson via bitcoin-dev]

Humour me please,


Where you say "create transactions which are only valid if they are mined on 
top of a specific block." - in practice, does that usually means at any height 
above a specific block?


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
 on behalf of Ruben Somsen via 
bitcoin-dev 
Sent: Sunday, 2 September 2018 3:26:54 AM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: [bitcoin-dev] Guiding transaction fees towards a more censorship 
resistant outcome

When a user creates a transaction with a fee attached, they are
incentivizing miners to add this transaction to the blockchain. The
task is usually not very specific -- as long as it ends up in a valid
chain with the most Proof-of-Work, miners get paid. The payment is an
incentive for miners to act in the way that users desire.

To the user, there’s an individual benefit: their transaction gets
added. To the network, there’s a shared benefit: all fees add to the
security of other transactions in the chain. Miners can choose to
ignore the incentives, but they would be leaving money on the table
(and eventually get replaced by more competitive miners).

Transactions from Bitcoin Core are slightly more specific about what
they ask miners to do. Every transaction is only valid at a block
height that is one higher than the last block. This incentivizes
miners to build on top of the last block, instead of going back and
reorganizing the blockchain. This is especially important in a future
scenario where the fee reward is larger than the block reward.

BIP 115* by Luke-jr is even more specific. It enables users to create
transactions which are only valid if they are mined on top of a
specific block. While originally designed as a form of replay
protection, it actually serves as a deterrent for miners to reorganize
the blockchain. If they orphan a block, it will invalidate
transactions that demanded the inclusion of the orphaned block. This
increases the cost of intentionally reorganizing the blockchain.

Coinjoin**, the act of combining payments of multiple users into a
single transaction, can be seen as yet another method for users to be
more specific. The fate of their payments are now intertwined with
that of others. If miners wish to censor a single payment, they have
to reject the entire transaction, and the associated fee amount.
Techniques like mimblewimble simplify this process, by making coinjoin
non-interactive.

This brings us to a theoretical scenario where:

- every block contains only a single coinjoin transaction
- the validity of this transaction depends on the inclusion of a
specific previous block
- the block reward is negligible compared to transaction fees

In this scenario, if miners wish to undo a specific transaction that
happened two blocks ago, they would have to create three empty blocks
(receiving negligible compensation) in order to overtake the longest
chain. And even then, users can still refuse to let their new
transactions be mined on top of the empty blocks, disincentivizing
such behavior completely.

While not easy to achieve in practice (e.g. resolving natural forks
becomes more complicated), it demonstrates that users can become more
empowered than they are today, benefitting censorship resistance***.
It is this line of thinking that I wish to convey. Perhaps it may
inspire further ideas in this direction.

-- Ruben Somsen


* BIP 115: 
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbitcoin%2Fbips%2Fblob%2Fmaster%2Fbip-0115.mediawikidata=02%7C01%7C%7C48403f77efc14613e89b08d611f5980c%7C84df9e7fe9f640afb435%7C1%7C0%7C636716143839246019sdata=req4KYOcztXLAG%2Fu4RrmhLREGBF28JNTe45pO86kRd4%3Dreserved=0

** Coinjoin: 
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbitcointalk.org%2Findex.php%3Ftopic%3D279249.0data=02%7C01%7C%7C48403f77efc14613e89b08d611f5980c%7C84df9e7fe9f640afb435%7C1%7C0%7C636716143839246019sdata=d%2B06jxrKubWhLwoInFEgo8eHvI9f1j74QN8WH7xrVos%3Dreserved=0

*** Risk sharing principle:
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flibbitcoin%2Flibbitcoin%2Fwiki%2FRisk-Sharing-Principledata=02%7C01%7C%7C48403f77efc14613e89b08d611f5980c%7C84df9e7fe9f640afb435%7C1%7C0%7C636716143839246019sdata=NA3HxqI5PnuyaI9hyCaw0rcaFsrhD%2FXQB8biWJXej8g%3Dreserved=0
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.linuxfoundation.org%2Fmailman%2Flistinfo%2Fbitcoin-devdata=02%7C01%7C%7C48403f77efc14613e89b08d611f5980c%7C84df9e7fe9f640afb435%7C1%7C0%7C636716143839246019sdata=9P7UetPmKWngjgjNPE0%2BAMgdzuL2DgqBLoLti82f23M%3Dreserved=0
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] [bitcoin-discuss] Checkpoints in the Blockchain.

[Damian Williamson via bitcoin-dev]

I do understand your point, however, 'something like stuxnet' cannot be used to 
create valid data without re-doing all the PoW. Provided some valid copies of 
the blockchain continue to exist, the network can re-synchronise.


Unrelated, it would seem useful to have some kind of deep blockchain corruption 
recovery mechanism if it does not exist; where blocks are altered at a depth 
exceeding the re-scan on init, efficient recovery is possible on detection. 
Presumably, it would be possible for some stuxnet like thing to modify blocks 
by modifying the table data making blocks invalid but without causing a table 
corruption. I would also suppose that if the node is delving deep into the 
blockchain for transaction data, that is would also validate the block at least 
that it has a valid hash (apart from Merkle tree validation for the individual 
transaction?) and that the hash of its immediate ancestor is also valid.


Regards,

Damian Williamson



From: bitcoin-discuss-boun...@lists.linuxfoundation.org 
 on behalf of Dave Scotese 
via bitcoin-discuss 
Sent: Sunday, 20 May 2018 11:58 AM
To: Scott Roberts
Cc: Bitcoin Discuss
Subject: Re: [bitcoin-discuss] Checkpoints in the Blockchain.

I wouldn't limit my rendering to words, but that is a decent starting point.  
The richer the rendering, the harder it will be to forget, but it needn't all 
be developed at once. My goal here is to inspire the creation of art that is, 
at the same time, highly useful and based on randomness.

Anyway, I asked what "premise that this is needed" you meant and I still don't 
know the answer.

"The archive is a shared memory" - yes, a shared computer memory, and growing 
larger (ie more cumbersome) every day. If something like stuxnet is used to 
change a lot of the copies of it at some point, it seems likely that people 
will notice a change, but which history is correct won't be so obvious because 
for the humans whose memories are not so easily overwritten, computer data is 
remarkably non-memorable in it's normal form (0-9,a-f, whatever).  If we ever 
want to abandon the historical transaction data, having a shared memory of the 
state of a recent UTXO Set will help to obviate the need for it.  Yes, of 
course the blockchain is the perfect solution, as long as there is exactly one 
and everyone can see that it's the same one that everyone else sees.  Any other 
number of archives presents a great difficulty.

In that scenario, there's no other way to prove that the starting point is 
valid.  Bitcoin has included a hardcoded-checkpoint in the code which served 
the same purpose, but this ignores the possibility that two versions of the 
code could be created, one with a fake checkpoint that is useful to a powerful 
attacker.  If the checkpoint were rendered into something memorable at the 
first opportunity, there would be little question about which one is correct 
when the difference is discovered.

On Sat, May 19, 2018 at 5:22 PM, Scott Roberts 
> wrote:
I just don't see the point of needing to know it any different from the hex 
value. Or maybe I should say I can't imagine it being useful because I can't 
imagine what you're after is possible. There might be a theoretical proof that 
what you're after is impossible. Hard to forget is almost the opposite of many 
options and what we're trying to do is decide between many options. I'll assume 
English because  it's the only starting point I have that's even in the 
ballpark of being possible. You might need to constrain the word selection and 
the structure in which you put it. I can only imagine that you are talking 
about putting the words into some sort of story. The only kind of story that 
would be hard to forget  is one that  fits into an overall structure that we 
are familiar with but those types of structures are few  compared to the 
possibilities that we're trying to encode. "Hard to deny" is a type of "hard to 
forget". Besides trying to connect it to external reality or history that we 
can all agree on there is also an internal consistency that could be used like 
a checksum such as the structure I mentioned. The only thing that seems to fit 
the bill is the blockchain itself. It's on everyone's computer so it's 
impossible to forget and it has internal consistency. Is the only shared memory 
we have that can't be subject to a Sybil attack or other hijacking of our 
memory of the true history. Our translation of the hash into words and a story 
could potentially be subject to hijacking if it's not done perfectly correct. 
It just seems best to me to use the hash itself. They archived existence of the 
prior parts of the blockchain are what make that particular hash hard to 
forget. Supposedly it can't be forged to reference  a fake history. The archive 
is a shared memory that fits 

Re: [bitcoin-dev] feature: Enhance privacy by change obfuscation

[Damian Williamson via bitcoin-dev]

I note that Electrum v3.0.6 has an option to use multiple change addresses. It 
is off by default.


Regards,

Damian Williamson


From: Eric Voskuil 
Sent: Monday, 19 March 2018 5:59:28 AM
To: Evan Klitzke; Bitcoin Protocol Discussion
Cc: Damian Williamson
Subject: Re: [bitcoin-dev] feature: Enhance privacy by change obfuscation

> This would be really expensive for the network due to the bloat in UTXO size, 
> a cost everyone has to pay for.

Without commenting on the merits of this proposal, I’d just like to correct 
this common misperception. There is no necessary additional cost to the network 
from the count of unspent outputs. This perception arises from an 
implementation detail of particular node software. There is no requirement for 
redundant indexing of unspent outputs.

e
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] feature: Enhance privacy by change obfuscation

[Damian Williamson via bitcoin-dev]

Alright, but even if two (or more) of the change outputs were linked in a 
future transaction, no-one can tell if they are still linked to your wallet or 
not unless there is also an additional re-used address on the new transaction 
input side that has also been previously linked to one of the inputs on the 
transaction creating the change.


Yes, I understand the additional cost but still thought it worthy of 
consideration.


Regards,

Damian Williamson


From: Evan Klitzke <e...@eklitzke.org>
Sent: Sunday, 18 March 2018 4:50:34 PM
To: Damian Williamson; Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] feature: Enhance privacy by change obfuscation


Damian Williamson via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> writes:
> Operation: Provide a user selectable 'Enhanced privacy' option for
> transaction creation, when true the transaction randomly distributes
> change across up to twenty output addresses (minimum five?), provided
> each output is not dust.

This would be really expensive for the network due to the bloat in UTXO
size, a cost everyone has to pay for. Not to mention the fact that it
doesn't really seem that private, as the wallet is likely going to have
to rejoin those inputs in future transactions (and the user will have to
pay a high transaction fee as a result).

--
Evan Klitzke
https://eklitzke.org/
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] feature: Enhance privacy by change obfuscation

[Damian Williamson via bitcoin-dev]

Application: Bitcoin Core

Feature: Enhanced privacy by change obfuscation

Operation: Provide a user selectable 'Enhanced privacy' option for transaction 
creation, when true the transaction randomly distributes change across up to 
twenty output addresses (minimum five?), provided each output is not dust.

Suggestions: Perhaps limit the total random number of addresses to distribute 
to by change amount. Optionally: If necessary, additional inputs can be 
selected if available to increase change although consider if this may 
eventually result in a decrease in obfuscation in some cases when the outputs 
are spent.

Issues: Transaction cost will be higher for the initial spend with the change 
due to increased outputs and, possibly for later spending the change depending 
on the future spend amount(s) and the number of inputs required.

Argument: If transaction linkage is possible, it is still possible with the 
obfuscated change but, it is far more difficult to guess what was retained by 
the owner of the originating utxo's unless the new change outputs are spent 
together in the same transaction.


Regards,

Damian Williamson
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] {sign|verify}message replacement

[Damian Williamson via bitcoin-dev]

That is very helpful Luke. I would not have been concerned if it was necessary 
to sign multiple times for multiple utxo's on different addresses but, since it 
is a feature it may as well be best usable. Signing for multiple inputs 
verifying that you have the priv key for each in your wallet is certainly 
usable for this popular misuse.


>Ideally, it should support not only just "proof I receive at this address",
but also "proof of funds" (as a separate feature) since this is a popular
misuse of the current message signing (which doesn't actually prove funds at
all). To do this, it needs to be capable of signing for multiple inputs.


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
 on behalf of Luke Dashjr via 
bitcoin-dev 
Sent: Wednesday, 14 March 2018 11:36:47 PM
To: Karl Johan Alm; Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] {sign|verify}message replacement

I don't see a need for a new RPC interface, just a new signature format.

Ideally, it should support not only just "proof I receive at this address",
but also "proof of funds" (as a separate feature) since this is a popular
misuse of the current message signing (which doesn't actually prove funds at
all). To do this, it needs to be capable of signing for multiple inputs.

Preferably, it should also avoid disclosing the public key for existing or
future UTXOs. But I don't think it's possible to avoid this without something
MAST-like first. Perhaps it can be a MAST upgrade later on, but the new
signature scheme should probably be designed with it in mind.

Luke


On Wednesday 14 March 2018 8:09:20 AM Karl Johan Alm via bitcoin-dev wrote:
> Hello,
>
> I am considering writing a replacement for the message signing tools
> that are currently broken for all but the legacy 1xx addresses. The
> approach (suggested by Pieter Wuille) is to do a script based
> approach. This does not seem to require a lot of effort for
> implementing in Bitcoin Core*. Below is my proposal for this system:
>
> A new structure SignatureProof is added, which is a simple scriptSig &
> witnessProgram container that can be serialized. This is passed out
> from/into the signer/verifier.
>
> RPC commands:
>
> sign   [=false]
>
> Generates a signature proof for  using the same method that
> would be used to spend coins sent to .**
>
> verify[=false]
>
> Deserializes and executes the proof using a custom signature checker
> whose sighash is derived from . Returns true if the check
> succeeds, and false otherwise. The scriptPubKey is derived directly
> from .**
>
> Feedback welcome.
>
> -Kalle.
>
> (*) Looks like you can simply use VerifyScript with a new signature
> checker class. (h/t Nicolas Dorier)
> (**) If  is true,  is the sighash, otherwise
> sighash=sha256d(message).
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Sign / Verify message against SegWit P2SH and Bech32 addresses

[Damian Williamson via bitcoin-dev]

Current implementation of sign/verify is broken for SegWit and Bech32 addresses.


Please add the following reference to the use cases:

---

# Does blockchain.info show balances for addresses that are in cold storage?

Yes.

>... is there any way for me in another country to confirm that what my 
>colleague views is actually accurate and correct?

Since they use Bitcoin Core, yes, there is a way to verify that they hold the 
addresses that they claim. Have them sign a message with each address that they 
claim to have the holdings on, using Bitcoin Core you can verify that they 
indeed have those addresses and check them on blockchain.info to find the 
current balance.

Only works in Bitcoin Core currently for addresses starting with a '1' (not 
Segwit addresses starting with a '3' and not Bech32 addresses starting with 
'bc1' - the developers are aware of this and I will remind them shortly.)

In Bitcoin Core, your transaction opposite goes to File -> Sign Message and 
signs any message with one of the holding addresses. Copy the message, address 
and signature and send to you via probably plain text format email is the 
easiest. Repeat for each additional address holding the balance of BTC that 
they are offering to sell.

In Bitcoin Core, you go to File -> Verify Message and key the details provided 
EXACTLY - spaces, new lines and all characters must be an EXACT match. Click on 
verify and voilà.

I prefer the form of signed message as follows (don't key the top and bottom 
bar rows for the message, just the contents and you can check this yourself, 
the bottom row is the signature). I like to key the address used for verifying 
as a part of the message but that is not strictly necessary:

--
Something that I want to sign.

bitcoin:1PMUf9aaQ41M4bgVbCAPVwAeuKvj8CwxJg
--
Signture:

IGaXlQNRHHM6ferJ+Ocr3cN9dRJhIWxo+n9PGwgg1uPdOLVYIeCuaccEzDygVgYPJMXqmQeSaLaZVoG6FMHPJkg=

This contains all of the compact information necessary to verify the message.

Example of verified message:
![verified message][1]

[1]: https://i.stack.imgur.com/zv1xq.png

---

https://bitcoin.stackexchange.com/a/72281/75001



Solution seems to be straight-forward, as noted in Issue# 
[10542](https://github.com/bitcoin/bitcoin/issues/10542#issuecomment-306584383)


>And it would in theory be possible to make signmessage work for a P2SH-P2WPKH 
>address, in cases where the verifier knows the embedded pubkeyhash already. 
>But in that case you don't need "sign with a witness address" functionality - 
>*you could just sign with the embedded key (see validateaddress), and have the 
>verifier check that*.


>The point is to not further the misunderstanding that signmessage signs with 
>an address - it never did. It signs with a keyhash, and verify with a keyhash.


This is an important feature, there are few other ways to verify that an 
address is held. Note that the linked issue is not currently labeld GUI and 
probably could be - unless a new issue should also be opened?


Regards,

Damian Williamson

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP Activation Reference

[Damian Williamson via bitcoin-dev]

Would it be possible or desirable to add a `nBlockHeight Activated` column to 
the 
[README.mediawiki](https://github.com/bitcoin/bips/blob/master/README.mediawiki)
 to show a specific reference to when a BIP was activated? - And/or include 
such information in the BIP Header format?


Regards,

Damian Williamson
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Some thoughts on removing timestamps in PoW

[Damian Williamson via bitcoin-dev]

>1. Introducing state checkpoints into the chain itself could make it possible 
>for full nodes to skip verification of large sections of historical data when 
>booting up.


What you are suggesting, unless I am mistaken, is that new full nodes should 
have no way of knowing if an output is spent or even if it exists. Since large 
sections of the blockchain will potentially be skipped, the full node will not 
have complete knowledge of utxo's just for starters.


Regards,

Damian Williamson


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
 on behalf of Tao Effect via 
bitcoin-dev 
Sent: Monday, 19 February 2018 12:29:50 PM
To: Bitcoin Protocol Discussion
Subject: [bitcoin-dev] Some thoughts on removing timestamps in PoW

Copied from: 
https://github.com/WebOfTrustInfo/rebooting-the-web-of-trust-spring2018/pull/13


# Blockchain Timestamps Unnecessary In Proof-of-Work?

*Author: Greg Slepak 
([@taoeffect@mastodon.social](https://mastodon.social/@taoeffect))*



The Bitcoin blockchain has a 10-minute target blocktime that is achieved by a 
difficulty adjustment algorithm.

I assert, or rather, pose the hypothesis, that the use of timestamps in 
Bitcoin's blockchain may be unnecessary, and that Bitcoin can operate with the 
same security guarantees without it (except as noted in [Risks and 
Mitigations](#risks-and-mitigations)), and therefore does not need miners to 
maintain global clock synchronization.

The alternative difficulty adjustment algorithm would work according to the 
following principles:

- The incentive for miners is and always has been to maximize profit.
- The block reward algorithm is now modified to issue coins into perpetuity (no 
maximum). Any given block can issue _up to_ `X` number of coins per block.
- The number of coins issued per block is now tied directly to the difficulty 
of the block, and the concept of "epocs" or "block reward halving" is removed.
- The chain selection rule remains "chain with most proof of work"
- The difficulty can be modified by miners in an arbitrary direction (up or 
down), but is limited in magnitude by some maximum percentage (e.g. no more 
than 20% deviation from the previous block), we call this `Y%`.

### Observations

- Miners are free to mine blocks of whatever difficulty they choose, up to a 
maximum deviation
- The blockchain may at times produce blocks very quickly, and at other times 
produce blocks more slowly
- Powerful miners are incentivized to raise the difficulty to remove 
competitors (as is true today)
- Whether miners choose to produce blocks quickly or slowly is entirely up to 
them. If they produce blocks quickly, each block has a lower reward, but there 
are more of them. If they produce blocks slowly, each block has a higher 
reward, but there are fewer of them. So an equilibrium will be naturally 
reached to produce blocks at a rate that should minimize orphans.

A timestamp may still be included in blocks, but it no longer needs to be used 
for anything, or represent anything significant other than metadata about when 
the miner claims to have produced the block.

### Risks and Mitigations

Such a system may introduce risks that require further modification of the 
protocol to mitigate.

The most straightforward risk comes from the potential increase in total 
transaction throughput that such a change would introduce (these are the same 
concerns that exist with respect to raising the blocksize). The removal of 
timestamps would allow a cartel of miners to produce high-difficulty blocks at 
a fast rate, potentially resulting in additional centralization pressures not 
only on miners but also on full nodes who then would have greater difficulty 
keeping up with the additional bandwidth and storage demands.

Two equally straightforward mitigations exist to address this if we are given 
the liberty of modifying the protocol as we wish:

1. Introducing state checkpoints into the chain itself could make it possible 
for full nodes to skip verification of large sections of historical data when 
booting up.
2. A sharded protocol, where each shard uses a "sufficiently different" PoW 
algorithm, would create an exit for users should the primary blockchain become 
captured by a cartel providing poor quality-of-service.


--
Please do not email me anything that you are not comfortable also sharing with 
the NSA.

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Possible change to the MIT license

[Damian Williamson via bitcoin-dev]

I do not know that Bitcoin's position is any weaker because of the terms that 
the software is licenced under.


Cory Fields said:

>Let other projects faff about with copyright litigation and trademark dilution 
>concerns


I disagree completely with any licence change. As well as allowing for the use 
of a software, a licence is also a disclaimer for those responsible for the 
release. Changing a single word can have drastic implications should there ever 
be any action considered against any involved party or group. The current MIT 
licence is IMHO the right fit.


Regards,

Damian Williamson
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] NIST 8202 Blockchain Technology Overview

[Damian Williamson via bitcoin-dev]

Then you have my apology, I will not claim to be any kind of advocate or user 
of Bitcoin Cash but *had* understood that segwith had been enabled. Clearly my 
mistake.


Regards,

Damian Williamson


From: CANNON 
Sent: Tuesday, 6 February 2018 1:08:24 PM
To: Damian Williamson; Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] NIST 8202 Blockchain Technology Overview

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 01/31/2018 11:16 AM, Damian Williamson wrote:
> I disagree with the correctness of the following statement:
>
>
>> Rather than implementing the SegWit changes, the developers of Bitcoin Cash 
>> decided to simply increase the blocksize.
>
>
> I would suggest "Rather than being satisfied with the implementation of 
> SegWit changes alone, the developers of
> Bitcoin Cash decided to also increase the blocksize.
>
>
> Regards,
>
> Damian Williamson

You do realize that segwit includes many improvements of which are unrelated to 
scaling? These same improvements of which
simply increasing the blocksize alone would not fix or enable. Segwit is not 
just a blocksize increase.
Bitcoin Cash, while increasing the blocksize directly, from my understanding 
has yet to implement the
improvements and capabilities that segwit enables.

One example being, with transactions hashes being able to be calculated in 
advanced prior to signing
(due to the signature being in different section than that used
to calculate the transaction ID) it is possible to create transaction trees, 
enhanced smart contracts, trustless mixing protocols,
micropayment networks, etc...

Segwit also increases the security of signatures.

There are lots of other things segregated witness enables as well.

By saying "..segwit changes alone decided to also..." Bitcoin Cash has not 
implemented segwit. Bitcoin Cash only
increased the blocksize.

that wording above at least from the way I read it, seems to imply that Bitcoin 
Cash has segwit.
-BEGIN PGP SIGNATURE-

iQIcBAEBCgAGBQJaeQ3IAAoJEAYDai9lH2mwKkQP/3dgYApq1qv2lGIyZIdeN9SE
D5AuXPqFQYAoMwhC0RPNQU/jUisKIyd6zm4XCIm6KPufCtXkjfH9FLhd0ThbCTcy
Gk+pYYRBzSuBZdPBKg0DHu7alRETtxbdtUI0zDfERt1FFZb+HmcDcGTfwdVci3fa
jBiFXq1R+myMW5xdB44dipSk5kBhcpx2zitr1bIA4rF11QbxKAmzU7iPdRpA+PXz
gB9NImc1Dbz+TEA50tdq3v9Ov3x7m7F+QtBnqyLAigJh6XKa6guCfwKIGoawRGwZ
v2ur7T+Qh3KGRXCBlHnxgtFte16wHagwvsVgE5EEmJR0yJUc/4XU2kCGANVNDZ/P
pphqk8pruQ5rjQ8S+s6i5XG8oHVSB2fDh56NvPY7msA72j+Gk+XneV2eJbEAdjhb
9Ci7u1uPJL3pb3c/ZOwQvpIRV3tRjlh0DertWkd3Li5RZLO3uFvBTxNxrni6+9bf
/cmAOwfHjoUp8BX/nvgMjpIDCoEu+Rv9IO/ok3s3mX300JbczAdGbXbsPTE5G+DI
RB1kSmszwst8wOlOAsdVqk/iKRJdN9daTGGN6aE/wjkpSg8rW9BOaoI2X9t4oXCU
+oe/WlgkxhxPcNyhKpLeeYVe6nFX2fjU+THyyiAq/LJ/qHU/brKpXc4NesCVHhQP
BBlxiN0E4gndMGs/Lx89
=+UCK
-END PGP SIGNATURE-

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] Does Lightning require millisatoshi unit?

[Damian Williamson via bitcoin-dev]

It seems in a document that I was referenced with this very question that the 
unit for creating invoices on Lightning is millisatoshi. Do we really need to 
invoice for 1000 millisatoshi for a 1 sat transaction?


https://github.com/ElementsProject/lightning/blob/master/README.md#sending-and-receiving-payments

[https://avatars2.githubusercontent.com/u/12729539?s=400=4]

lightning/README.md at master · ElementsProject/lightning 
...
github.com
c-lightning is a standard compliant implementation of the Lightning Network 
protocol. The Lightning Network is a scalability solution for Bitcoin, enabling 
secure and ...


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Good afternoon Alan,


It is stated in the proposal that it is intended for blocks to be validated as 
the output of the priority method, to ensure that they conform. Unfortunately, 
the math necessary for this sort of statistical function is outside the scope 
of my formal education and I will need to rely on someone to develop what is 
necessary. If it does turn out that this is not ultimately possible then I 
suppose at that stage the proposal would need to be abandoned since I agree - 
validation must be necessary. Blocks created with cheating should be too 
unlikely.


>All block created with dynamic size should be verified to ensure
conformity to a probability distribution curve resulting from the
priority method. Since the input is a probability, the output should
conform to a probability distribution.


Regards,

Damian Williamson


From: Alan Evans <thealanev...@gmail.com>
Sent: Sunday, 21 January 2018 1:46:41 AM
To: Damian Williamson; Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

I don't see any modifications to the proposal that addresses the issue that 
miners will always be free to choose their own priority that a few people 
brought up before.

I understand you think it's in the miners best long-term interest to follow 
these rules, but even if a miner agrees with you, if that miner thinks the 
other miners are following the fee curve, they will know it makes no overall 
difference if they cheat (you can't prove how long a miner has had a 
transaction in their mempool).

The opportunity to cheat, the anonymity of mining, the low negative effect of a 
single cheating instance, all combined with a financial incentive to cheat 
means that cheating will be rife.


On Sat, Jan 20, 2018 at 8:04 AM, Damian Williamson via bitcoin-dev 
<bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>>
 wrote:

Tried a different approach for the curves, would appreciate it if someone has 
the energy to work on this and help me to resolve it a bit more scientifically:


p(tx) = (fx - (fl - 0.0001)) / (fh - (fl - 0.0001))) * 100) + 1) ^ 
y) + (wx - 0.9) / ((86400 * n) - 0.9)) * 100) + 1) ^ y)

p is the calculated priority number for tx the specific valid transaction.
fx is the fee in BTC/KB for the specific transaction.
fl is the lowest valid fee in BTC/KB currently in the nodes mempool.
fh is the highest valid fee in BTC/KB currently in the nodes mempool.
wx is the current wait in seconds for tx the specific valid transaction.
n is the number of days maximum wait consensus value.
y can be 10 or, y can be a further developed to be a formula based on the 
number of required inclusions to vary the steepness of the curve as the mempool 
size varies.

In the next step, the random value must be:
if random(101^y) < p then transaction is included;

Regards,
Damian Williamson



From: Damian Williamson <willt...@live.com.au<mailto:willt...@live.com.au>>
Sent: Saturday, 20 January 2018 10:25:43 AM
To: Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks


An example curve:

The curve curently described here is ineffective at acheiving the requirements. 
It seems to be not nearly steep enough resulting in too many inclusions (as it 
happens, this may not metter - needs further evaluation) and, the lower end 
values seem problematically small but, results in a number between 100 for the 
highest fee BTC/KB and a small fraction of 1 for the lowest. This math needs to 
be improved.


pf(tx) = sin2((fx-(fl-0.0001))/(fh-(fl-0.0001))*1.570796326795)*100


pf is the calculated priority number for the fee for tx the specifc valid 
transaction.
fx is the fee in BTC/KB for the specific transaction.
fl is the lowest valid fee in BTC/KB currently in the nodes mempool.
fh is the highest valid fee in BTC/KB currently in the nodes mempool.


From: 
bitcoin-dev-boun...@lists.linuxfoundation.org<mailto:bitcoin-dev-boun...@lists.linuxfoundation.org>
 
<bitcoin-dev-boun...@lists.linuxfoundation.org<mailto:bitcoin-dev-boun...@lists.linuxfoundation.org>>
 on behalf of Damian Williamson via bitcoin-dev 
<bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>>
Sent: Thursday, 4 January 2018 8:01:10 PM
To: Bitcoin Protocol Discussion
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks


This proposal has a new update, mostly minor edits. Additionally, I had a logic 
flaw in the hard fork / soft fork declaration statement. The specific terms of 
the CC-BY-SA-4.0 licence the document is published under have now been updated 
to include 

Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Tried a different approach for the curves, would appreciate it if someone has 
the energy to work on this and help me to resolve it a bit more scientifically:


p(tx) = (fx - (fl - 0.0001)) / (fh - (fl - 0.0001))) * 100) + 1) ^ 
y) + (wx - 0.9) / ((86400 * n) - 0.9)) * 100) + 1) ^ y)

p is the calculated priority number for tx the specific valid transaction.
fx is the fee in BTC/KB for the specific transaction.
fl is the lowest valid fee in BTC/KB currently in the nodes mempool.
fh is the highest valid fee in BTC/KB currently in the nodes mempool.
wx is the current wait in seconds for tx the specific valid transaction.
n is the number of days maximum wait consensus value.
y can be 10 or, y can be a further developed to be a formula based on the 
number of required inclusions to vary the steepness of the curve as the mempool 
size varies.

In the next step, the random value must be:
if random(101^y) < p then transaction is included;

Regards,
Damian Williamson



From: Damian Williamson <willt...@live.com.au>
Sent: Saturday, 20 January 2018 10:25:43 AM
To: Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks


An example curve:

The curve curently described here is ineffective at acheiving the requirements. 
It seems to be not nearly steep enough resulting in too many inclusions (as it 
happens, this may not metter - needs further evaluation) and, the lower end 
values seem problematically small but, results in a number between 100 for the 
highest fee BTC/KB and a small fraction of 1 for the lowest. This math needs to 
be improved.


pf(tx) = sin2((fx-(fl-0.0001))/(fh-(fl-0.0001))*1.570796326795)*100


pf is the calculated priority number for the fee for tx the specifc valid 
transaction.
fx is the fee in BTC/KB for the specific transaction.
fl is the lowest valid fee in BTC/KB currently in the nodes mempool.
fh is the highest valid fee in BTC/KB currently in the nodes mempool.


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson 
via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Thursday, 4 January 2018 8:01:10 PM
To: Bitcoin Protocol Discussion
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks


This proposal has a new update, mostly minor edits. Additionally, I had a logic 
flaw in the hard fork / soft fork declaration statement. The specific terms of 
the CC-BY-SA-4.0 licence the document is published under have now been updated 
to include additional permissions available under the MIT licence.


Recently, on Twitter:

I am looking for a capable analyst/programmer to work on a BIP proposal as 
co-author. Will need to format several Full BIP's per these BIP process 
requirements: ( https://github.com/bitcoin/bips/blob/master/bip-0002.mediawiki 
) from a BIP Proposal, being two initially for non-consensus full-interoperable 
pre-rollout on peer service layer & API/RPC layer and, a reference 
implementation for Bitcoin Core per: ( 
https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md ). Interested 
parties please reply via this list thread: ( 
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/015485.html
 ) #Bitcoin #BIP


Regards,

Damian Williamson



From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson 
via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Monday, 1 January 2018 10:04 PM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

Happy New Year all.

This proposal has been further amended with several minor changes and a
few additions.

I believe that all known issues raised so far have been sufficiently
addressed. Either that or, I still have more work to do.

## BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For
Ordering Transactions In Blocks

Schema:
##
Document: BIP Proposal
Title: UTPFOTIB - Use Transaction Priority For Ordering Transactions In
Blocks
Published: 26-12-2017
Revised: 01-01-2018
Author: Damian Williamson <willt...@live.com.au>
Licence: Creative Commons Attribution-ShareAlike 4.0 International
License.
URL: http://thekingjameshrmh.tumblr.com/post/168948530950/bip-proposal-
utpfotib-use-transaction-priority-for-order
##

### 1. Abstract

This document proposes to address the issue of transactional
reliability in Bitcoin, where valid transactions may be stuck in the
transaction pool for extended periods or never confirm.

There are two key issues to be resolved to achieve this:

1.  The current transact

Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

An example curve:

The curve curently described here is ineffective at acheiving the requirements. 
It seems to be not nearly steep enough resulting in too many inclusions (as it 
happens, this may not metter - needs further evaluation) and, the lower end 
values seem problematically small but, results in a number between 100 for the 
highest fee BTC/KB and a small fraction of 1 for the lowest. This math needs to 
be improved.


pf(tx) = sin2((fx-(fl-0.0001))/(fh-(fl-0.0001))*1.570796326795)*100


pf is the calculated priority number for the fee for tx the specifc valid 
transaction.
fx is the fee in BTC/KB for the specific transaction.
fl is the lowest valid fee in BTC/KB currently in the nodes mempool.
fh is the highest valid fee in BTC/KB currently in the nodes mempool.


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson 
via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Thursday, 4 January 2018 8:01:10 PM
To: Bitcoin Protocol Discussion
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks


This proposal has a new update, mostly minor edits. Additionally, I had a logic 
flaw in the hard fork / soft fork declaration statement. The specific terms of 
the CC-BY-SA-4.0 licence the document is published under have now been updated 
to include additional permissions available under the MIT licence.


Recently, on Twitter:

I am looking for a capable analyst/programmer to work on a BIP proposal as 
co-author. Will need to format several Full BIP's per these BIP process 
requirements: ( https://github.com/bitcoin/bips/blob/master/bip-0002.mediawiki 
) from a BIP Proposal, being two initially for non-consensus full-interoperable 
pre-rollout on peer service layer & API/RPC layer and, a reference 
implementation for Bitcoin Core per: ( 
https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md ). Interested 
parties please reply via this list thread: ( 
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/015485.html
 ) #Bitcoin #BIP


Regards,

Damian Williamson



From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson 
via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Monday, 1 January 2018 10:04 PM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

Happy New Year all.

This proposal has been further amended with several minor changes and a
few additions.

I believe that all known issues raised so far have been sufficiently
addressed. Either that or, I still have more work to do.

## BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For
Ordering Transactions In Blocks

Schema:
##
Document: BIP Proposal
Title: UTPFOTIB - Use Transaction Priority For Ordering Transactions In
Blocks
Published: 26-12-2017
Revised: 01-01-2018
Author: Damian Williamson <willt...@live.com.au>
Licence: Creative Commons Attribution-ShareAlike 4.0 International
License.
URL: http://thekingjameshrmh.tumblr.com/post/168948530950/bip-proposal-
utpfotib-use-transaction-priority-for-order
##

### 1. Abstract

This document proposes to address the issue of transactional
reliability in Bitcoin, where valid transactions may be stuck in the
transaction pool for extended periods or never confirm.

There are two key issues to be resolved to achieve this:

1.  The current transaction bandwidth limit.
2.  The current ad-hoc methods of including transactions in blocks
resulting in variable and confusing confirmation times for valid
transactions, including transactions with a valid fee that may never
confirm.

It is important with any change to protect the value of fees as these
will eventually be the only payment that miners receive. Rather than an
auction model for limited bandwidth, the proposal results in a fee for
priority service auction model.

It would not be true to suggest that all feedback received so far has
been entirely positive although, most of it has been constructive.

The previous threads for this proposal are available here:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/s
ubject.html

In all parts of this proposal, references to a transaction, a valid
transaction, a transaction with a valid fee, a valid fee, etc. is
defined as any transaction that is otherwise valid with a fee of at
least 0.1000 BTC/KB as defined as the dust level, interpreting from
Bitcoin Core GUI. Transactions with a fee lower than this rate are
considered dust.

In all parts of this proposal, dust and zero-fee transactions are
always ignored and/or excluded unless specifically mentioned.

It is generally assumed that 

Re: [bitcoin-dev] Plausible Deniability (Re: Satoshilabs secret shared private key scheme)

[Damian Williamson via bitcoin-dev]

The same problems exist for users of whole disk encrypted operating systems. 
Once the device (or, the initial password authentication) is found, the 
adversary knows that there is something to see. The objective of plausible 
deniability is to present some acceptable (plausible) alternative while keeping 
the actual hidden (denied).


If the adversary does not believe you, you do indeed risk everything.


Regards,

Damian Williamson


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
 on behalf of nullius via 
bitcoin-dev 
Sent: Friday, 12 January 2018 10:06:33 PM
To: Peter Todd; Bitcoin Protocol Discussion
Subject: [bitcoin-dev] Plausible Deniability (Re: Satoshilabs secret shared 
private key scheme)

On 2018-01-12 at 09:50:58 +, Peter Todd  wrote:
>On Tue, Jan 09, 2018 at 12:43:48PM +, Perry Gibson wrote:
>>>Trezor's "plausible deniability" scheme could very well result in you
>>>going to jail for lying to border security, because it's so easy for
>>>them to simply brute force alternate passwords based on your seeds.
>>>With that, they have proof that you lied to customs, a serious
>>>offense.
>>The passphrase scheme as I understand it allows a maximum of 50
>>characters to be used.  Surely even with the HD seed, that search
>>space is too large to brute force.  Or is there a weakness in the
>>scheme I haven't clocked?
>
>While passphrases *can* be long, most user's aren't going to understand
>the risk. For example, Trezors blog(1) doesn't make it clear that the
>passphrases could be bruteforced and used as evidence against you, and
>even suggests the contrary:  [...quote...]

I despise the term “plausible deniability”; and that’s really the wrong
term to use in this discussion.

“Plausible deniability” is a transparent excuse for explaining away an
indisputable fact which arouses suspicion—when you got some serious
’splain’ to do.  This is usually used in the context of some pseudolegal
argument about introducing “reasonable doubt”, or even making “probable
cause” a wee bit less probable.

“Why yes, officer:  I was seen carrying an axe down the street near the
site of an axe murder, at approximately the time of said axe murder.
But I do have a fireplace; so it is plausible that I was simply out
gathering wood.”

I rather suspect the concept of “plausible deniability” of having been
invented by a detective or agent provocateur.  There are few concepts
more useful for helping suspects shoot themselves in the foot, or
frankly, for entrapping people.

One of the worst examples I have seen is in discussions of Monero,
whereby I’ve seen proponents claim that even under the worst known
active attacks, their mix scheme reduces transaction linking to a
maximum of 20–40% probability.  “That’s not good enough to convince a
jury!”  No, but it is certainly adequate for investigators to identify
you as a person of interest.  Then, your (mis)deeds can be subjected to
powerful confirmation attacks based on other data; blockchains do not
exist in isolation.  I usually stay out of such discussions; for I have
no interest in helping the sorts of people whose greatest concern in
life is what story to foist on a jury.

In the context of devices such as Trezor, what is needed is not
“plausible deniability”, but rather the ability to obviate any need to
deny anything at all.  I must repeat, information does not exist in
isolation.

If you are publicly known to be deepy involved in Bitcoin, then nobody
will believe that your one-and-only wallet contains only 0.01 BTC.
That’s not even “plausible”.  But if you have overall privacy practices
which leave nobody knowing or suspecting that you have any Bitcoin at
all, then there is nothing to “deny”; and should a Trezor with
(supposedly) 0.01 BTC be found in your possession, that’s much better
than “plausible”.  It’s completely unremarkable.

Whereas if you are known or believed to own large amounts of BTC, a
realistic bad guy’s response to your “decoy” wallet could be, “I don’t
believe you; and it costs me nothing to keep beating you with rubber
hose until you tell me the *real* password.”

It could be worse, too.  In a kidnapping scenario, the bad guys could
say, “I don’t believe you.  Hey, I also read Trezor’s website about
‘plausible deniability’.  Now, I will maim your kid for life just to
test whether you told me the *real* password.  And if you still don’t
tell me the real password after you see that little Johnny can no longer
walk, then I will kill him.”

The worst part is that you have no means of proving that you really
*did* give the real password.  Indeed, it can be proved if you’re lying
by finding a password which reveals a hidden wallet—but *you* have no
means of affirmatively proving that you are telling the truth!  If the
bad guys overestimated your riches (or if they’re in a bad mood), then
little Johnny is dead 

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

This proposal has a new update, mostly minor edits. Additionally, I had a logic 
flaw in the hard fork / soft fork declaration statement. The specific terms of 
the CC-BY-SA-4.0 licence the document is published under have now been updated 
to include additional permissions available under the MIT licence.


Recently, on Twitter:

I am looking for a capable analyst/programmer to work on a BIP proposal as 
co-author. Will need to format several Full BIP's per these BIP process 
requirements: ( https://github.com/bitcoin/bips/blob/master/bip-0002.mediawiki 
) from a BIP Proposal, being two initially for non-consensus full-interoperable 
pre-rollout on peer service layer & API/RPC layer and, a reference 
implementation for Bitcoin Core per: ( 
https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md ). Interested 
parties please reply via this list thread: ( 
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/015485.html
 ) #Bitcoin #BIP


Regards,

Damian Williamson



From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson 
via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Monday, 1 January 2018 10:04 PM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

Happy New Year all.

This proposal has been further amended with several minor changes and a
few additions.

I believe that all known issues raised so far have been sufficiently
addressed. Either that or, I still have more work to do.

## BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For
Ordering Transactions In Blocks

Schema:
##
Document: BIP Proposal
Title: UTPFOTIB - Use Transaction Priority For Ordering Transactions In
Blocks
Published: 26-12-2017
Revised: 01-01-2018
Author: Damian Williamson <willt...@live.com.au>
Licence: Creative Commons Attribution-ShareAlike 4.0 International
License.
URL: http://thekingjameshrmh.tumblr.com/post/168948530950/bip-proposal-
utpfotib-use-transaction-priority-for-order
##

### 1. Abstract

This document proposes to address the issue of transactional
reliability in Bitcoin, where valid transactions may be stuck in the
transaction pool for extended periods or never confirm.

There are two key issues to be resolved to achieve this:

1.  The current transaction bandwidth limit.
2.  The current ad-hoc methods of including transactions in blocks
resulting in variable and confusing confirmation times for valid
transactions, including transactions with a valid fee that may never
confirm.

It is important with any change to protect the value of fees as these
will eventually be the only payment that miners receive. Rather than an
auction model for limited bandwidth, the proposal results in a fee for
priority service auction model.

It would not be true to suggest that all feedback received so far has
been entirely positive although, most of it has been constructive.

The previous threads for this proposal are available here:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/s
ubject.html

In all parts of this proposal, references to a transaction, a valid
transaction, a transaction with a valid fee, a valid fee, etc. is
defined as any transaction that is otherwise valid with a fee of at
least 0.1000 BTC/KB as defined as the dust level, interpreting from
Bitcoin Core GUI. Transactions with a fee lower than this rate are
considered dust.

In all parts of this proposal, dust and zero-fee transactions are
always ignored and/or excluded unless specifically mentioned.

It is generally assumed that miners currently prefer to include
transactions with higher fees.

### 2. The need for this proposal

We all must learn to admit that transaction bandwidth is still lurking
as a serious issue for the operation, reliability, safety, consumer
acceptance, uptake and, for the value of Bitcoin.

I recently sent a payment which was not urgent so; I chose three-day
target confirmation from the fee recommendation. That transaction has
still not confirmed after now more than six days - even waiting twice
as long seems quite reasonable to me (note for accuracy: it did
eventually confirm). That transaction is a valid transaction; it is not
rubbish, junk or, spam. Under the current model with transaction
bandwidth limitation, the longer a transaction waits, the less likely
it is ever to confirm due to rising transaction numbers and being
pushed back by transactions with rising fees.

I argue that no transactions with fees above the dust level are rubbish
or junk, only some zero fee transactions might be spam. Having an ever-
increasing number of valid transactions that do not confirm as more new
transactions with higher fees are created is the opposite of operating
a robust, reliable transaction system.

While the miners have 

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Happy New Year all.

This proposal has been further amended with several minor changes and a
few additions.

I believe that all known issues raised so far have been sufficiently
addressed. Either that or, I still have more work to do.

## BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For
Ordering Transactions In Blocks

Schema:  
##  
Document: BIP Proposal  
Title: UTPFOTIB - Use Transaction Priority For Ordering Transactions In
Blocks  
Published: 26-12-2017  
Revised: 01-01-2018  
Author: Damian Williamson   
Licence: Creative Commons Attribution-ShareAlike 4.0 International
License.  
URL: http://thekingjameshrmh.tumblr.com/post/168948530950/bip-proposal-
utpfotib-use-transaction-priority-for-order  
##

### 1. Abstract

This document proposes to address the issue of transactional
reliability in Bitcoin, where valid transactions may be stuck in the
transaction pool for extended periods or never confirm.

There are two key issues to be resolved to achieve this:

1.  The current transaction bandwidth limit.
2.  The current ad-hoc methods of including transactions in blocks
resulting in variable and confusing confirmation times for valid
transactions, including transactions with a valid fee that may never
confirm.

It is important with any change to protect the value of fees as these
will eventually be the only payment that miners receive. Rather than an
auction model for limited bandwidth, the proposal results in a fee for
priority service auction model.

It would not be true to suggest that all feedback received so far has
been entirely positive although, most of it has been constructive.

The previous threads for this proposal are available here:  
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/s
ubject.html

In all parts of this proposal, references to a transaction, a valid
transaction, a transaction with a valid fee, a valid fee, etc. is
defined as any transaction that is otherwise valid with a fee of at
least 0.1000 BTC/KB as defined as the dust level, interpreting from
Bitcoin Core GUI. Transactions with a fee lower than this rate are
considered dust.

In all parts of this proposal, dust and zero-fee transactions are
always ignored and/or excluded unless specifically mentioned.

It is generally assumed that miners currently prefer to include
transactions with higher fees.

### 2. The need for this proposal

We all must learn to admit that transaction bandwidth is still lurking
as a serious issue for the operation, reliability, safety, consumer
acceptance, uptake and, for the value of Bitcoin.

I recently sent a payment which was not urgent so; I chose three-day
target confirmation from the fee recommendation. That transaction has
still not confirmed after now more than six days - even waiting twice
as long seems quite reasonable to me (note for accuracy: it did
eventually confirm). That transaction is a valid transaction; it is not
rubbish, junk or, spam. Under the current model with transaction
bandwidth limitation, the longer a transaction waits, the less likely
it is ever to confirm due to rising transaction numbers and being
pushed back by transactions with rising fees.

I argue that no transactions with fees above the dust level are rubbish
or junk, only some zero fee transactions might be spam. Having an ever-
increasing number of valid transactions that do not confirm as more new
transactions with higher fees are created is the opposite of operating
a robust, reliable transaction system.

While the miners have discovered a gold mine, it is the service they
provide that is valuable. If the service is unreliable they are not
worth the gold that they mine. This is reflected in the value of
Bitcoin.

Business cannot operate with a model where transactions may or may not
confirm. Even a business choosing a modest fee has no guarantee that
their valid transaction will not be shuffled down by new transactions
to the realm of never confirming after it is created. Consumers also
will not accept this model as Bitcoin expands. If Bitcoin cannot be a
reliable payment system for confirmed transactions then consumers, by
and large, will simply not accept the model once they understand.
Bitcoin will be a dirty payment system, and this will kill the value of
Bitcoin.

Under the current system, a minority of transactions will eventually be
the lucky few who have fees high enough to escape being pushed down the
list.

Once there are more than x transactions (transaction bandwidth limit)
every ten minutes, only those choosing twenty-minute confirmation (2
blocks) from the fee recommendations will have initially at most a
fifty percent chance of ever having their payment confirm by the time
2x transactions is reached. Presently, not even using fee
recommendations can ensure a sufficiently high fee is paid to ensure
transaction confirmation.

I also argue that the current auction model for limited transaction
bandwidth is wrong, is 

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

I have needed to re-tac my intentions somewhat, there is still much
work to be done.

This is a request for assistance and further discussion of the re-
revised proposal. I am sure there are still issues to be resolved.

## BIP Proposal: UTPFOTIB - Use Transaction Priority For Ordering
Transactions In Blocks

Schema:  
##  
Document: BIP Proposal  
Title: UTPFOTIB - Use Transaction Priority For Ordering Transactions In
Blocks  
Date: 26-12-2017  
Author: Damian Williamson willt...@live.com.au  
Licence: Creative Commons Attribution-ShareAlike 4.0 International
License.  
URL: http://thekingjameshrmh.tumblr.com/post/168948530950/bip-proposal-
utpfotib-use-transaction-priority-for-order  
##  

### 1. Abstract

This document proposes to address the issue of transactional
reliability in Bitcoin, where valid transactions may be stuck in the
transaction pool for extended periods or never confirm.

There are two key issues to be resolved to achieve this:

1.  The current transaction bandwidth limit.
2.  The current ad-hoc methods of including transactions in blocks
resulting in variable and confusing confirmation times for valid
transactions, including transactions with a valid fee that may never
confirm.

It is important with any change to protect the value of fees as these
will eventually be the only payment that miners receive. Rather than an
auction model for limited bandwidth, the proposal results in a fee for
priority service auction model.

It would not be true to suggest that all feedback received so far has
been entirely positive although, most of it has been constructive.

The previous threads for this proposal are available here:  
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/s
ubject.html

In all parts of this proposal, references to a transaction, a valid
transaction, a transaction with a valid fee, a valid fee, etc. is
defined as any transaction that is otherwise valid with a fee of at
least 0.1000 BTC/KB as defined as the dust level, interpreting from
Bitcoin Core GUI. Transactions with a fee lower than this rate are
considered dust.

In all parts of this proposal, dust and zero-fee transactions are
always ignored and/or excluded unless specifically mentioned.

It is generally assumed that miners currently prefer to include
transactions with higher fees.

### 2. The need for this proposal

We all must learn to admit that transaction bandwidth is still lurking
as a serious issue for the operation, reliability, safety, consumer
acceptance, uptake and, for the value of Bitcoin.

I recently sent a payment which was not urgent so; I chose three-day
target confirmation from the fee recommendation. That transaction has
still not confirmed after now more than six days - even waiting twice
as long seems quite reasonable to me (note for accuracy: it did
eventually confirm). That transaction is a valid transaction; it is not
rubbish, junk or, spam. Under the current model with transaction
bandwidth limitation, the longer a transaction waits, the less likely
it is ever to confirm due to rising transaction numbers and being
pushed back by transactions with rising fees.

I argue that no transactions with fees above the dust level are rubbish
or junk, only some zero fee transactions might be spam. Having an ever-
increasing number of valid transactions that do not confirm as more new
transactions with higher fees are created is the opposite of operating
a robust, reliable transaction system.

Business cannot operate with a model where transactions may or may not
confirm. Even a business choosing a modest fee has no guarantee that
their valid transaction will not be shuffled down by new transactions
to the realm of never confirming after it is created. Consumers also
will not accept this model as Bitcoin expands. If Bitcoin cannot be a
reliable payment system for confirmed transactions then consumers, by
and large, will simply not accept the model once they understand.
Bitcoin will be a dirty payment system, and this will kill the value of
Bitcoin.

Under the current system, a minority of transactions will eventually be
the lucky few who have fees high enough to escape being pushed down the
list.

Once there are more than x transactions (transaction bandwidth limit)
every ten minutes, only those choosing twenty-minute confirmation (2
blocks) from the fee recommendations will have initially at most a
fifty percent chance of ever having their payment confirm when 2x
transactions is reached. Presently, not even using fee recommendations
can ensure a sufficiently high fee is paid to ensure transaction
confirmation.

I also argue that the current auction model for limited transaction
bandwidth is wrong, is not suitable for a reliable transaction system
and, is wrong for Bitcoin. All transactions with valid fees must
confirm in due time. Currently, Bitcoin is not a safe way to send
payments.

I do not believe that consumers and business are against paying fees,
even high 

Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

>.. This system has flaws, they all do.


>The simple fact is that there is currently no known system that works as well 
>as the current system..


Alright, but, we seem to agree, the current system also has flaws. The 
transaction bandwidth limit is a serious issue for transactional reliability.


What you have proposed is interesting but seems to do nothing for the issue of 
transaction bandwidth, which seems to be approaching its threshold:

https://bitinfocharts.com/comparison/bitcoin-transactions.html


Regards,

Damian Williamson


From: Spartacus Rex <spartacusre...@gmail.com>
Sent: Saturday, 23 December 2017 5:07:49 AM
To: Damian Williamson; Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

Hi Damian,

Thought I'd chip in.  This is a hard fork scenario. This system has flaws, they 
all do.

If you had a fixed fee per block, so that every txn in that block paid the same 
fee, that might make it easier to include all txns eventually, as you envisage.

The fee could be calculated as the average of the amount txns are prepared to 
pay in the last 1000 blocks.

A txn would say ' I'll pay up to X bitcoins ' and as long as that is more than 
the value required for the block your txn can be added. This is to ensure you 
don't pay more than you are willing.  It also ensures that putting an enormous 
fee will not ensure your txn is processed quickly..

Calculating what the outputs are given a variable fee needs a new mechanism all 
of it's own, but I'm sure it's possible.

The simple fact is that there is currently no known system that works as well 
as the current system..

But there are other systems.


On Dec 22, 2017 15:09, "Damian Williamson via bitcoin-dev" 
<bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>>
 wrote:

If the cash value of Bitcoin was high enough and zero fee transactions were 
never accepted and not counted when calculating the transaction pool size then 
I do not think it would be such an issue. Why is it even possible to create 
zero fee transactions?


Regards,

Damian Williamson


From: 
bitcoin-dev-boun...@lists.linuxfoundation.org<mailto:bitcoin-dev-boun...@lists.linuxfoundation.org>
 
<bitcoin-dev-boun...@lists.linuxfoundation.org<mailto:bitcoin-dev-boun...@lists.linuxfoundation.org>>
 on behalf of Damian Williamson via bitcoin-dev 
<bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>>
Sent: Tuesday, 19 December 2017 6:51 PM
To: Mark Friedenbach
Cc: 
bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks


Thank you for your constructive feedback. I now see that the proposal 
introduces a potential issue.


>Finally in terms of the broad goal, having block size based on the number of 
>transactions is NOT something desirable in the first place, even if it did 
>work. That’s effectively the same as an infinite block size since anyone 
>anywhere can create transactions in the mempool at no cost.


Do you have any critical suggestion as to how transaction bandwidth limit could 
be addressed, it will eventually become an issue if nothing is changed 
regardless of how high fees go?


Regards,

Damian Williamson




From: Mark Friedenbach <m...@friedenbach.org<mailto:m...@friedenbach.org>>
Sent: Tuesday, 19 December 2017 3:08 AM
To: Damian Williamson
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

Damian, you seem to be misunderstanding that either

(1) the strong form of your proposal requires validating the commitment to the 
mempool properties, in which case the mempool becomes consensus critical (an 
impossible requirement); or

(2) in the weak form where the current block is dependent on the commitment in 
the last block only it is becomes a miner-selected field they can freely 
parameterize with no repercussions for setting values totally independent of 
the actual mempool.

If you want to make the block size dependent on the properties of the mempool 
in a consensus critical way, flex cap achieves this. If you want to make the 
contents or properties of the mempool known to well-connected nodes, weak 
blocks achieves that. But you can’t stick the mempool in consensus because it 
fundamentally is not something the nodes have consensus over. That’s a 
chicken-and-the-egg assumption.

Finally in terms of the broad goal, having block size based on the number of 
transactions is NOT something desirable in the first place, even if it did 
work. That’s effectively the same as an infinite block size since

[bitcoin-dev] BIP 177: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

BIP 177: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks


This BIP proposes to address the issue of transactional reliability in Bitcoin, 
where valid transactions may be stuck in the mempool for extended periods.


There are two key issues to be resolved:


  1.  The current transaction bandwidth limit.
  2.  The current ad-hoc methods of including transactions in blocks resulting 
in variable and confusing confirmation times for valid transactions, including 
transactions with a valid fee that may never confirm.


It is important with any change to protect the value of fees as these will 
eventually be the only payment that miners receive. Rather than an auction 
model for limited bandwidth, the proposal results in a stable fee for priority 
service auction model.


I will post the full proposal up on to my blog in the coming days and, 
re-review incorporating feedback that I have received on and off thread. It 
would not be true to suggest that all feedback received has been entirely 
positive although, most of it has been constructive.


The previous threads for this BIP are available here:

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/subject.html


Regards,

Damian Williamson
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

I suppose what I intended is (2) the weak form but, what is essentially needed 
is (1) the strong form. The answer may be somewhere in-between.


I do not see that an entire consensus for the mempool is needed, each node just 
needs a loose understanding of the average number of non-zero fee transactions 
in the mempool.


As a pre-rollout, it would be possible to give each node a serial ID and, 
calculate the average number of non-zero fee transactions from the information 
it has and, say every ten minutes, distribute information it has about the 
number of transactions in the mempool. Each node would be able to form its own 
picture of the average number of non-zero fee transactions in the mempool.


At rollout, this information would be the basis a node would use when a block 
is solved to provide the next expected block size. This would still not stop 
cheating by providing especially a number lower than the proposal would allow 
for, to game the system and hike fees. If miners will not act in the long-term 
interest of the stability and operation of the system then they should be 
ignored. If most miners will adhere to the proposal then the average effect 
would be stability in the operation of the proposal, having a few or even 
several nodes posting low numbers for the number of transactions expected in 
the next expected block size would not destroy the operation. If some node 
posted an insanely high number for next expected block size that resulted in 
the mempool being emptied then the proposal would be offended but I do not 
actually care. If no number is posted, just create a block the appropriate size 
ensure conformity. Nodes that have not adopted the proposal could just continue 
to create 1MB blocks.


Actually, the operation could be simplified using the distributed information 
directly to just create blocks of the appropriate size with no need to provide 
next block size. Flexible block size.


The proposal should also specify a minimum number of transactions to include 
for the next block to give at a minimum a 1MB block.


I currently have no information on flex cap, do you have a link?


Regards,

Damian Williamson



From: Mark Friedenbach <m...@friedenbach.org>
Sent: Tuesday, 19 December 2017 3:08 AM
To: Damian Williamson
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

Damian, you seem to be misunderstanding that either

(1) the strong form of your proposal requires validating the commitment to the 
mempool properties, in which case the mempool becomes consensus critical (an 
impossible requirement); or

(2) in the weak form where the current block is dependent on the commitment in 
the last block only it is becomes a miner-selected field they can freely 
parameterize with no repercussions for setting values totally independent of 
the actual mempool.

If you want to make the block size dependent on the properties of the mempool 
in a consensus critical way, flex cap achieves this. If you want to make the 
contents or properties of the mempool known to well-connected nodes, weak 
blocks achieves that. But you can’t stick the mempool in consensus because it 
fundamentally is not something the nodes have consensus over. That’s a 
chicken-and-the-egg assumption.

Finally in terms of the broad goal, having block size based on the number of 
transactions is NOT something desirable in the first place, even if it did 
work. That’s effectively the same as an infinite block size since anyone 
anywhere can create transactions in the mempool at no cost.

On Dec 16, 2017, at 8:14 PM, Damian Williamson via bitcoin-dev 
<bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>>
 wrote:

I do not know why people make the leap that the proposal requires a consensus 
on the transaction pool. It does not.

It may be helpful to have the discussion from the previous thread linked here.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/015370.html

Where I speak of validating that a block conforms to the broadcast next block 
size, I do not propose validating the number broadcast for the next block size 
itself, only that the next generated block is that size.

Regards,
Damian Williamson



From: Damian Williamson <willt...@live.com.au<mailto:willt...@live.com.au>>
Sent: Saturday, 16 December 2017 7:59 AM
To: Rhavar
Cc: Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

There are really two separate problems to solve.


  1.  How does Bitcoin scale with fixed block size?
  2.  How do we ensure that all valid transactions are eventually included in 
the blockchain?

Those are the two issues that the proposal attempts to address. It makes sense 
to resolve these two problems toget

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

If the cash value of Bitcoin was high enough and zero fee transactions were 
never accepted and not counted when calculating the transaction pool size then 
I do not think it would be such an issue. Why is it even possible to create 
zero fee transactions?


Regards,

Damian Williamson


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson 
via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Tuesday, 19 December 2017 6:51 PM
To: Mark Friedenbach
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks


Thank you for your constructive feedback. I now see that the proposal 
introduces a potential issue.


>Finally in terms of the broad goal, having block size based on the number of 
>transactions is NOT something desirable in the first place, even if it did 
>work. That’s effectively the same as an infinite block size since anyone 
>anywhere can create transactions in the mempool at no cost.


Do you have any critical suggestion as to how transaction bandwidth limit could 
be addressed, it will eventually become an issue if nothing is changed 
regardless of how high fees go?


Regards,

Damian Williamson




From: Mark Friedenbach <m...@friedenbach.org>
Sent: Tuesday, 19 December 2017 3:08 AM
To: Damian Williamson
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

Damian, you seem to be misunderstanding that either

(1) the strong form of your proposal requires validating the commitment to the 
mempool properties, in which case the mempool becomes consensus critical (an 
impossible requirement); or

(2) in the weak form where the current block is dependent on the commitment in 
the last block only it is becomes a miner-selected field they can freely 
parameterize with no repercussions for setting values totally independent of 
the actual mempool.

If you want to make the block size dependent on the properties of the mempool 
in a consensus critical way, flex cap achieves this. If you want to make the 
contents or properties of the mempool known to well-connected nodes, weak 
blocks achieves that. But you can’t stick the mempool in consensus because it 
fundamentally is not something the nodes have consensus over. That’s a 
chicken-and-the-egg assumption.

Finally in terms of the broad goal, having block size based on the number of 
transactions is NOT something desirable in the first place, even if it did 
work. That’s effectively the same as an infinite block size since anyone 
anywhere can create transactions in the mempool at no cost.

On Dec 16, 2017, at 8:14 PM, Damian Williamson via bitcoin-dev 
<bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>>
 wrote:

I do not know why people make the leap that the proposal requires a consensus 
on the transaction pool. It does not.

It may be helpful to have the discussion from the previous thread linked here.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/015370.html

Where I speak of validating that a block conforms to the broadcast next block 
size, I do not propose validating the number broadcast for the next block size 
itself, only that the next generated block is that size.

Regards,
Damian Williamson



From: Damian Williamson <willt...@live.com.au<mailto:willt...@live.com.au>>
Sent: Saturday, 16 December 2017 7:59 AM
To: Rhavar
Cc: Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

There are really two separate problems to solve.


  1.  How does Bitcoin scale with fixed block size?
  2.  How do we ensure that all valid transactions are eventually included in 
the blockchain?

Those are the two issues that the proposal attempts to address. It makes sense 
to resolve these two problems together. Using the proposed system for variable 
block sizes would solve the first problem but there would still be a whole 
bunch of never confirming transactions. I am not sure how to reliably solve the 
second problem at scale without first solving the first.

>* Every node has a (potentially) different mempool, you can't use it to decide 
>consensus values like the max block size.

I do not suggest a consensus. Depending on which node solves a block the value 
for next block size will be different. The consensus would be that blocks will 
adhere to the next block size value transmitted with the current block. It is 
easy to verify that the consensus is being adhered to once in place.

>* Increasing the entropy in a block to make it more unpredictable doesn't 
>really make sense.

Not

[bitcoin-dev] A DNS-like decentralized mapping for wallet addresses?

[Damian Williamson via bitcoin-dev]

There is no reason it should not be easily possible to develop a Bitcoin wallet 
that has an integrated name to address mapping feature. It might be a good idea 
for a software product, it could even be based on Bitcoin Core. There is no 
specific reason that people wanting that sort of feature could not use it. In 
fact, you could map names, strings, email addresses, it could be very flexible.


Relying on an additional service like DNS which is flexible enough to handle 
the job, does introduce an additional availability risk. There is no additional 
privacy risk provided each mapped name or address is only used once to 
send/receive one payment unless you directly use something personally 
identifiable like an email address which could be used to map bitcoin addresses 
to an individual. Personally, I am not concerned about privacy so much but can 
understand that some highly value their privacy.


If you get it right it will be a service better than namecoin transacting in 
Bitcoin. If you think that is valuable, go for it.


Regards,

Damian Williamson



From: bitcoin-dev-boun...@lists.linuxfoundation.org 
 on behalf of Sjors Provoost via 
bitcoin-dev 
Sent: Monday, 18 December 2017 10:26 PM
To: Douglas Roark; Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] A DNS-like decentralized mapping for wallet 
addresses?

Have you thought about combining this with BIP-47? You could associate payment 
codes with email via DNS.

It would be nice if there was a way to get rid of the announcement transaction 
in BIP-47 and establish a shared secret out of bound. That would simplify 
things, at the cost of an additional burden of storing more than an HD seed to 
recover a wallet that received funds this way.

Perhaps the sender can email to the recipient the information they need to 
retrieve the funds. The (first) transaction could have a time locked refund in 
it, in case the payment code is stale.

Sjors

> Op 1 dec. 2017, om 04:08 heeft Douglas Roark via bitcoin-dev 
>  het volgende geschreven:
>
> On 2017/11/30 14:20, mandar mulherkar via bitcoin-dev wrote:
>> I was wondering in terms of mass adoption, instead of long wallet
>> addresses, maybe there should be a DNS-like decentralized mapping
>> service to provide a user@crypto address?
>
> A few years ago, I was part of an effort with Armory and Verisign to
> make something similar to what you're describing.
> https://tools.ietf.org/html/draft-wiley-paymentassoc-00 is where you can
> find the one and only official draft. I worked on a follow-up with some
> changes and some nice appendices, explaining some nice tricks one could
> use to make payment management flexible. For various reasons, it never
> got published. I think it's an interesting draft that could be turned
> into something useful. Among other things, it was able to leverage BIP32
> and allow payment requests to be generated that automatically pointed
> payees to the correct branch. DNSSEC may have some issues but, AFAIK,
> it's as the easiest way to bootstrap identity to a common, reasonably
> secure standard.
>
> --
> ---
> Douglas Roark
> Cryptocurrency, network security, travel, and art.
> https://onename.com/droark
> joro...@vt.edu
> PGP key ID: 26623924
>
> ___
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Thank you for your constructive feedback. I now see that the proposal 
introduces a potential issue.


>Finally in terms of the broad goal, having block size based on the number of 
>transactions is NOT something desirable in the first place, even if it did 
>work. That’s effectively the same as an infinite block size since anyone 
>anywhere can create transactions in the mempool at no cost.


Do you have any critical suggestion as to how transaction bandwidth limit could 
be addressed, it will eventually become an issue if nothing is changed 
regardless of how high fees go?


Regards,

Damian Williamson




From: Mark Friedenbach <m...@friedenbach.org>
Sent: Tuesday, 19 December 2017 3:08 AM
To: Damian Williamson
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

Damian, you seem to be misunderstanding that either

(1) the strong form of your proposal requires validating the commitment to the 
mempool properties, in which case the mempool becomes consensus critical (an 
impossible requirement); or

(2) in the weak form where the current block is dependent on the commitment in 
the last block only it is becomes a miner-selected field they can freely 
parameterize with no repercussions for setting values totally independent of 
the actual mempool.

If you want to make the block size dependent on the properties of the mempool 
in a consensus critical way, flex cap achieves this. If you want to make the 
contents or properties of the mempool known to well-connected nodes, weak 
blocks achieves that. But you can’t stick the mempool in consensus because it 
fundamentally is not something the nodes have consensus over. That’s a 
chicken-and-the-egg assumption.

Finally in terms of the broad goal, having block size based on the number of 
transactions is NOT something desirable in the first place, even if it did 
work. That’s effectively the same as an infinite block size since anyone 
anywhere can create transactions in the mempool at no cost.

On Dec 16, 2017, at 8:14 PM, Damian Williamson via bitcoin-dev 
<bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>>
 wrote:

I do not know why people make the leap that the proposal requires a consensus 
on the transaction pool. It does not.

It may be helpful to have the discussion from the previous thread linked here.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/015370.html

Where I speak of validating that a block conforms to the broadcast next block 
size, I do not propose validating the number broadcast for the next block size 
itself, only that the next generated block is that size.

Regards,
Damian Williamson



From: Damian Williamson <willt...@live.com.au<mailto:willt...@live.com.au>>
Sent: Saturday, 16 December 2017 7:59 AM
To: Rhavar
Cc: Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

There are really two separate problems to solve.


  1.  How does Bitcoin scale with fixed block size?
  2.  How do we ensure that all valid transactions are eventually included in 
the blockchain?

Those are the two issues that the proposal attempts to address. It makes sense 
to resolve these two problems together. Using the proposed system for variable 
block sizes would solve the first problem but there would still be a whole 
bunch of never confirming transactions. I am not sure how to reliably solve the 
second problem at scale without first solving the first.

>* Every node has a (potentially) different mempool, you can't use it to decide 
>consensus values like the max block size.

I do not suggest a consensus. Depending on which node solves a block the value 
for next block size will be different. The consensus would be that blocks will 
adhere to the next block size value transmitted with the current block. It is 
easy to verify that the consensus is being adhered to once in place.

>* Increasing the entropy in a block to make it more unpredictable doesn't 
>really make sense.

Not a necessary function, just an effect of using a probability-based 
distribution.

>* Bitcoin should be roughly incentive compatible. Your proposal explicits asks 
>miners to ignore their best interests, and confirm transactions by "priority". 
> What are you going to do if a "malicious" miner decides to go after their 
>profits and order by what makes them the most money. Add "ordered by priority" 
>as a consensus requirement? And even if you miners can still sort their 
>mempool by fee, and then order the top 1MB by priority.

I entirely agree with your sentiment that Bitcoin must be incentive compatible. 
It is necessary.

It is in only miners immediate interest to make the most profitable b

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Thank you for your constructive feedback. I now see that the proposal 
introduces a potential issue.


It is difficult to define then, what is a valid transaction? Clearly, my 
definition was insufficient.


Regards,

Damian Williamson



From: Chris Riley <cri...@gmail.com>
Sent: Monday, 18 December 2017 11:09 PM
To: Damian Williamson; Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks

Regarding "problem" #2 where you say "How do we ensure that all valid 
transactions are eventually included in the blockchain?":  I do not believe 
that all people would (a) agree this is a problem or (b) that we do want to 
*ENSURE* that *ALL* valid transactions are eventually included in the 
blockchain.  There are many *valid* transactions that oftentimes miners do not 
(and should not) wish to require be confirmed and included in the blockchain.  
Spam transactions for example can be valid, but used to attack bitcoin by using 
no or low fee.  Any valid transaction MAY be included by a miner, but requiring 
it in some fashion at this point would open the network to other attack 
vectors.  Perhaps you meant it a different way.


On Fri, Dec 15, 2017 at 3:59 PM, Damian Williamson via bitcoin-dev 
<bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>>
 wrote:
>
> There are really two separate problems to solve.
>
>
> How does Bitcoin scale with fixed block size?
> How do we ensure that all valid transactions are eventually included in the 
> blockchain?
>
>
> Those are the two issues that the proposal attempts to address. It makes 
> sense to resolve these two problems together. Using the proposed system for 
> variable block sizes would solve the first problem but there would still be a 
> whole bunch of never confirming transactions. I am not sure how to reliably 
> solve the second problem at scale without first solving the first.
>
>
> >* Every node has a (potentially) different mempool, you can't use it to 
> >decide consensus values like the max block size.
>
>
> I do not suggest a consensus. Depending on which node solves a block the 
> value for next block size will be different. The consensus would be that 
> blocks will adhere to the next block size value transmitted with the current 
> block. It is easy to verify that the consensus is being adhered to once in 
> place.
>
> >* Increasing the entropy in a block to make it more unpredictable doesn't 
> >really make sense.
>
> Not a necessary function, just an effect of using a probability-based 
> distribution.
>
> >* Bitcoin should be roughly incentive compatible. Your proposal explicits 
> >asks miners to ignore their best interests, and confirm transactions by 
> >"priority".  What are you going to do if a "malicious" miner decides to go 
> >after their profits and order by what makes them the most money. Add 
> >"ordered by priority" as a consensus requirement? And even if you miners can 
> >still sort their mempool by fee, and then order the top 1MB by priority.
>
> I entirely agree with your sentiment that Bitcoin must be incentive 
> compatible. It is necessary.
>
> It is in only miners immediate interest to make the most profitable block 
> from the available transaction pool. As with so many other things, it is 
> necessary to partially ignore short-term gain for long-term benefit. It is in 
> miners and everybody's long-term interest to have a reliable transaction 
> service. A busy transaction service that confirms lots of transactions per 
> hour will become more profitable as demand increases and more users are 
> prepared to pay for priority. As it is there is currently no way to fully 
> scale because of the transaction bandwidth limit and that is problematic. If 
> all valid transactions must eventually confirm then there must be a way to 
> resolve that problem.
>
> Bitcoin deliberately removes traditional scale by ensuring blocks take ten 
> minutes on average to solve, an ingenious idea and, incentive compatible but, 
> fixed block sizes leaves us with a problem to solve when we want to scale.
>
> >If you could find a good solution that would allow you to know if miners 
> >were following your rule or not (and thus ignore it if it doesn't) then you 
> >wouldn't even need bitcoin in the first place.
>
> I am confident that the math to verify blocks based on the proposal can be 
> developed (and I think it will not be too complex for a mathematician with 
> the relevant experience), however, I am nowhere near experienced enough with 
> probability and statistical analysis to do it. Yes,

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

improvements =)

But just some quick notes:

* Every node has a (potentially) different mempool, you can't use it to decide 
consensus values like the max block size.

* Increasing the entropy in a block to make it more unpredictable doesn't 
really make sense.

* Bitcoin should be roughly incentive compatible. Your proposal explicits asks 
miners to ignore their best interests, and confirm transactions by "priority".  
What are you going to do if a "malicious" miner decides to go after their 
profits and order by what makes them the most money. Add "ordered by priority" 
as a consensus requirement? And even if you miners can still sort their mempool 
by fee, and then order the top 1MB by priority.

If you could find a good solution that would allow you to know if miners were 
following your rule or not (and thus ignore it if it doesn't) then you wouldn't 
even need bitcoin in the first place.




-Ryan


 Original Message 
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks
Local Time: December 15, 2017 3:42 AM
UTC Time: December 15, 2017 9:42 AM
From: bitcoin-dev@lists.linuxfoundation.org
To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>




I should not take it that the lack of critical feedback to this revised 
proposal is a glowing endorsement. I understand that there would be technical 
issues to resolve in implementation, but, are there no fundamental errors?

I suppose that it if is difficult to determine how long a transaction has been 
waiting in the pool then, each node could simply keep track of when a 
transaction was first seen. This may have implications for a verify routine, 
however, for example, if a node was offline, how should it differentiate how 
long each transaction was waiting in that case? If a node was restarted daily 
would it always think that all transactions had been waiting in the pool less 
than one day If each node keeps the current transaction pool in a file and 
updates it, as transactions are included in blocks and, as new transactions 
appear in the pool, then that would go some way to alleviate the issue, apart 
from entirely new nodes. There should be no reason the contents of a 
transaction pool files cannot be shared without agreement as to the transaction 
pool between nodes, just as nodes transmit new transactions freely.

It has been questioned why miners could not cheat. For the question of how many 
transactions to include in a block, I say it is a standoff and miners will 
conform to the proposal, not wanting to leave transactions with valid fees 
standing, and, not wanting to shrink the transaction pool. In any case, if 
miners shrink the transaction pool then I am not immediately concerned since it 
provides a more efficient service. For the question of including transactions 
according to the proposal, I say if it is possible to keep track of how long 
transactions are waiting in the pool so that they can be included on a 
probability curve then it is possible to verify that blocks conform to the 
proposal, since the input is a probability, the output should conform to a 
probability curve.



If someone has the necessary skill, would anyone be willing to develop the math 
necessary for the proposal?

Regards,
Damian Williamson


____

From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson 
via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Friday, 8 December 2017 8:01 AM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks



Good afternoon,

The need for this proposal:

We all must learn to admit that transaction bandwidth is still lurking as a 
serious issue for the operation, reliability, safety, consumer acceptance, 
uptake and, for the value of Bitcoin.

I recently sent a payment which was not urgent so; I chose three-day target 
confirmation from the fee recommendation. That transaction has still not 
confirmed after now more than six days - even waiting twice as long seems quite 
reasonable to me. That transaction is a valid transaction; it is not rubbish, 
junk or, spam. Under the current model with transaction bandwidth limitation, 
the longer a transaction waits, the less likely it is ever to confirm due to 
rising transaction numbers and being pushed back by transactions with rising 
fees.

I argue that no transactions are rubbish or junk, only some zero fee 
transactions might be spam. Having an ever-increasing number of valid 
transactions that do not confirm as more new transactions with higher fees are 
created is the opposite of operating a robust, reliable transaction system.

Business cannot operate with a model where transactions may or may not confirm. 
Even a b

Re: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

ing your rule or not (and thus ignore it if it doesn't) then you wouldn't 
even need bitcoin in the first place.




-Ryan


 Original Message 
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks
Local Time: December 15, 2017 3:42 AM
UTC Time: December 15, 2017 9:42 AM
From: bitcoin-dev@lists.linuxfoundation.org
To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>




I should not take it that the lack of critical feedback to this revised 
proposal is a glowing endorsement. I understand that there would be technical 
issues to resolve in implementation, but, are there no fundamental errors?

I suppose that it if is difficult to determine how long a transaction has been 
waiting in the pool then, each node could simply keep track of when a 
transaction was first seen. This may have implications for a verify routine, 
however, for example, if a node was offline, how should it differentiate how 
long each transaction was waiting in that case? If a node was restarted daily 
would it always think that all transactions had been waiting in the pool less 
than one day If each node keeps the current transaction pool in a file and 
updates it, as transactions are included in blocks and, as new transactions 
appear in the pool, then that would go some way to alleviate the issue, apart 
from entirely new nodes. There should be no reason the contents of a 
transaction pool files cannot be shared without agreement as to the transaction 
pool between nodes, just as nodes transmit new transactions freely.

It has been questioned why miners could not cheat. For the question of how many 
transactions to include in a block, I say it is a standoff and miners will 
conform to the proposal, not wanting to leave transactions with valid fees 
standing, and, not wanting to shrink the transaction pool. In any case, if 
miners shrink the transaction pool then I am not immediately concerned since it 
provides a more efficient service. For the question of including transactions 
according to the proposal, I say if it is possible to keep track of how long 
transactions are waiting in the pool so that they can be included on a 
probability curve then it is possible to verify that blocks conform to the 
proposal, since the input is a probability, the output should conform to a 
probability curve.



If someone has the necessary skill, would anyone be willing to develop the math 
necessary for the proposal?

Regards,
Damian Williamson


____

From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson 
via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Friday, 8 December 2017 8:01 AM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks



Good afternoon,

The need for this proposal:

We all must learn to admit that transaction bandwidth is still lurking as a 
serious issue for the operation, reliability, safety, consumer acceptance, 
uptake and, for the value of Bitcoin.

I recently sent a payment which was not urgent so; I chose three-day target 
confirmation from the fee recommendation. That transaction has still not 
confirmed after now more than six days - even waiting twice as long seems quite 
reasonable to me. That transaction is a valid transaction; it is not rubbish, 
junk or, spam. Under the current model with transaction bandwidth limitation, 
the longer a transaction waits, the less likely it is ever to confirm due to 
rising transaction numbers and being pushed back by transactions with rising 
fees.

I argue that no transactions are rubbish or junk, only some zero fee 
transactions might be spam. Having an ever-increasing number of valid 
transactions that do not confirm as more new transactions with higher fees are 
created is the opposite of operating a robust, reliable transaction system.

Business cannot operate with a model where transactions may or may not confirm. 
Even a business choosing a modest fee has no guarantee that their valid 
transaction will not be shuffled down by new transactions to the realm of never 
confirming after it is created. Consumers also will not accept this model as 
Bitcoin expands. If Bitcoin cannot be a reliable payment system for confirmed 
transactions then consumers, by and large, will simply not accept the model 
once they understand. Bitcoin will be a dirty payment system, and this will 
kill the value of Bitcoin.

Under the current system, a minority of transactions will eventually be the 
lucky few who have fees high enough to escape being pushed down the list.

Once there are more than x transactions (transaction bandwidth limit) every ten 
minutes, only those choosing twenty-minute confirmation (2 blocks) will have 
initially at most a fifty percen

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

I should not take it that the lack of critical feedback to this revised 
proposal is a glowing endorsement. I understand that there would be technical 
issues to resolve in implementation, but, are there no fundamental errors?

I suppose that it if is difficult to determine how long a transaction has been 
waiting in the pool then, each node could simply keep track of when a 
transaction was first seen. This may have implications for a verify routine, 
however, for example, if a node was offline, how should it differentiate how 
long each transaction was waiting in that case? If a node was restarted daily 
would it always think that all transactions had been waiting in the pool less 
than one day If each node keeps the current transaction pool in a file and 
updates it, as transactions are included in blocks and, as new transactions 
appear in the pool, then that would go some way to alleviate the issue, apart 
from entirely new nodes. There should be no reason the contents of a 
transaction pool files cannot be shared without agreement as to the transaction 
pool between nodes, just as nodes transmit new transactions freely.

It has been questioned why miners could not cheat. For the question of how many 
transactions to include in a block, I say it is a standoff and miners will 
conform to the proposal, not wanting to leave transactions with valid fees 
standing, and, not wanting to shrink the transaction pool. In any case, if 
miners shrink the transaction pool then I am not immediately concerned since it 
provides a more efficient service. For the question of including transactions 
according to the proposal, I say if it is possible to keep track of how long 
transactions are waiting in the pool so that they can be included on a 
probability curve then it is possible to verify that blocks conform to the 
proposal, since the input is a probability, the output should conform to a 
probability curve.


If someone has the necessary skill, would anyone be willing to develop the math 
necessary for the proposal?

Regards,
Damian Williamson


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Damian Williamson 
via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Friday, 8 December 2017 8:01 AM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: [bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction 
Priority For Ordering Transactions In Blocks


Good afternoon,

The need for this proposal:

We all must learn to admit that transaction bandwidth is still lurking as a 
serious issue for the operation, reliability, safety, consumer acceptance, 
uptake and, for the value of Bitcoin.

I recently sent a payment which was not urgent so; I chose three-day target 
confirmation from the fee recommendation. That transaction has still not 
confirmed after now more than six days - even waiting twice as long seems quite 
reasonable to me. That transaction is a valid transaction; it is not rubbish, 
junk or, spam. Under the current model with transaction bandwidth limitation, 
the longer a transaction waits, the less likely it is ever to confirm due to 
rising transaction numbers and being pushed back by transactions with rising 
fees.

I argue that no transactions are rubbish or junk, only some zero fee 
transactions might be spam. Having an ever-increasing number of valid 
transactions that do not confirm as more new transactions with higher fees are 
created is the opposite of operating a robust, reliable transaction system.

Business cannot operate with a model where transactions may or may not confirm. 
Even a business choosing a modest fee has no guarantee that their valid 
transaction will not be shuffled down by new transactions to the realm of never 
confirming after it is created. Consumers also will not accept this model as 
Bitcoin expands. If Bitcoin cannot be a reliable payment system for confirmed 
transactions then consumers, by and large, will simply not accept the model 
once they understand. Bitcoin will be a dirty payment system, and this will 
kill the value of Bitcoin.

Under the current system, a minority of transactions will eventually be the 
lucky few who have fees high enough to escape being pushed down the list.

Once there are more than x transactions (transaction bandwidth limit) every ten 
minutes, only those choosing twenty-minute confirmation (2 blocks) will have 
initially at most a fifty percent chance of ever having their payment confirm. 
Presently, not even using fee recommendations can ensure a sufficiently high 
fee is paid to ensure transaction confirmation.

I also argue that the current auction model for limited transaction bandwidth 
is wrong, is not suitable for a reliable transaction system and, is wrong for 
Bitcoin. All transactions must confirm in due time. Currently, Bitcoin is not a 
safe way to send payments.

I do not believe that consumers 

[bitcoin-dev] BIP Proposal: Revised: UTPFOTIB - Use Transaction Priority For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Good afternoon,

The need for this proposal:

We all must learn to admit that transaction bandwidth is still lurking as a 
serious issue for the operation, reliability, safety, consumer acceptance, 
uptake and, for the value of Bitcoin.

I recently sent a payment which was not urgent so; I chose three-day target 
confirmation from the fee recommendation. That transaction has still not 
confirmed after now more than six days - even waiting twice as long seems quite 
reasonable to me. That transaction is a valid transaction; it is not rubbish, 
junk or, spam. Under the current model with transaction bandwidth limitation, 
the longer a transaction waits, the less likely it is ever to confirm due to 
rising transaction numbers and being pushed back by transactions with rising 
fees.

I argue that no transactions are rubbish or junk, only some zero fee 
transactions might be spam. Having an ever-increasing number of valid 
transactions that do not confirm as more new transactions with higher fees are 
created is the opposite of operating a robust, reliable transaction system.

Business cannot operate with a model where transactions may or may not confirm. 
Even a business choosing a modest fee has no guarantee that their valid 
transaction will not be shuffled down by new transactions to the realm of never 
confirming after it is created. Consumers also will not accept this model as 
Bitcoin expands. If Bitcoin cannot be a reliable payment system for confirmed 
transactions then consumers, by and large, will simply not accept the model 
once they understand. Bitcoin will be a dirty payment system, and this will 
kill the value of Bitcoin.

Under the current system, a minority of transactions will eventually be the 
lucky few who have fees high enough to escape being pushed down the list.

Once there are more than x transactions (transaction bandwidth limit) every ten 
minutes, only those choosing twenty-minute confirmation (2 blocks) will have 
initially at most a fifty percent chance of ever having their payment confirm. 
Presently, not even using fee recommendations can ensure a sufficiently high 
fee is paid to ensure transaction confirmation.

I also argue that the current auction model for limited transaction bandwidth 
is wrong, is not suitable for a reliable transaction system and, is wrong for 
Bitcoin. All transactions must confirm in due time. Currently, Bitcoin is not a 
safe way to send payments.

I do not believe that consumers and business are against paying fees, even high 
fees. What is required is operational reliability.

This great issue needs to be resolved for the safety and reliability of 
Bitcoin. The time to resolve issues in commerce is before they become great big 
issues. The time to resolve this issue is now. We must have the foresight to 
identify and resolve problems before they trip us over.  Simply doubling block 
sizes every so often is reactionary and is not a reliable permanent solution. I 
have written a BIP proposal for a technical solution but, need your help to 
write it up to an acceptable standard to be a full BIP.

I have formatted the following with markdown which is human readable so, I hope 
nobody minds. I have done as much with this proposal as I feel that I am able 
so far but continue to take your feedback.

# BIP Proposal: UTPFOTIB - Use Transaction Priority For Ordering Transactions 
In Blocks

## The problem:
Everybody wants value. Miners want to maximize revenue from fees (and we 
presume, to minimize block size). Consumers need transaction reliability and, 
(we presume) want low fees.

The current transaction bandwidth limit is a limiting factor for both. As the 
operational safety of transactions is limited, so is consumer confidence as 
they realize the issue and, accordingly, uptake is limited. Fees are 
artificially inflated due to bandwidth limitations while failing to provide a 
full confirmation service for all transactions.

Current fee recommendations provide no satisfaction for transaction reliability 
and, as Bitcoin scales, this will worsen.

Bitcoin must be a fully scalable and reliable service, providing full 
transaction confirmation for every valid transaction.

The possibility to send a transaction with a fee lower than one that is 
acceptable to allow eventual transaction confirmation should be removed from 
the protocol and also from the user interface.

## Solution summary:
Provide each transaction with an individual transaction priority each time 
before choosing transactions to include in the current block, the priority 
being a function of the fee paid (on a curve), and the time waiting in the 
transaction pool (also on a curve) out to n days (n=60 ?). The transaction 
priority to serve as the likelihood of a transaction being included in the 
current block, and for determining the order in which transactions are tried to 
see if they will be included.

Use a target block size. Determine the target block size using; current 

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Good morning ZmnSCPxj,


Actually, there is no incentive to cheat target block size by providing a next 
block size that is higher or lower than the proposal would give. Under the 
proposal the transaction pool can grow quite large. A low next block size just 
defers collecting transaction fees, while a high next block size shrinks the 
transaction pool and thereby lowers fees. It seems like a standoff. This is 
especially true if the curve for time waiting in the transaction pool is 
extended beyond n days, since it is a curve, after waiting longer than 60 days 
(if n = 60 days) a transaction would have a priority greater than one-hundred 
and would therfore be the first transaction included with no possibility of 
failing the likelihood, so, even low fee paying transactions would be included 
first if the pool size is growing through incorrectly providing the next block 
size.


As it is now, I presume, a miner could include exactly one transaction in a 
block and pad?


Regards,

Damian Williamson


From: Damian Williamson 
Sent: Thursday, 7 December 2017 7:13:14 PM
To: ZmnSCPxj
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For 
Ordering Transactions In Blocks


Good morning ZmnSCPxj, it must be where you are,


I suppose that we are each missing each other's point some.


I understand that nodes would not be expected to agree on the transaction pool 
and do not propose validating that the correct transactions are included in a 
block. I speak of probability and likelihood of a transaction being included in 
a block, implying a random element. I do not propose rejecting blocks on the 
basis that the next block size is stated too large or too small for the 
transaction pool, only that the block received conforms to the next block size 
given on the previous block. Yes, it could be cheated. Also, various nodes may 
have at times wildly different amounts of transactions waiting in the 
transaction pool compared to each other and there could be a great disparity 
between them. It would not be possible in any case I can think of to validate 
the next block size is correct for the current transaction pool. Even as it is 
now, nodes may include transactions in a block that no other nodes have even 
heard of, nodes have no way to validate that either. If the block is built on 
sufficiently, it is the blockchain.


I will post back the revised proposal to the list. I have fleshed parts of it 
out more, given more explanation and, tried this time not to recycle 
terminology.


Regards,

Damian Williamson


From: ZmnSCPxj 
Sent: Thursday, 7 December 2017 5:46:08 PM
To: Damian Williamson
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For 
Ordering Transactions In Blocks

Good morning Damian,

>As I understand it, each node would be aware independently of x transactions 
>waiting for confirmation, the transaction pool.

Each long-running node would have a view that is roughly the same as the view 
of every other long-running node.

However, suppose a node, Sleeping Beauty, was temporarily stopped for a day 
(for various reasons) then is started again.  That node cannot verify what the 
"consensus" transaction pool was during the time it was stopped -- it has no 
view of that.  It can only trust that the longest chain is valid -- but that 
means it is SPV for this particular rule.

>If next blocksize is broadcast with the completed block it would be a simple 
>matter to back confirm that.

It would not. Suppose Sleeping Beauty slept at block height 500,000.  On 
awakening, some node provides some purported block at height 500,001.  This 
block indicates some "next blocksize" for the block at height 500,002.  How 
does Sleeping Beauty know that the transaction pool at block 500,001 was of the 
correct size to provide the given "next blocksize"?  The only way, would be to 
look if there is some other chain with greater height that includes or does not 
include that block: that is, SPV confirmation.

How does Sleeping Beauty know it has caught up, and that its transaction pool 
is similar to that of its neighbors (who might be lying to it, for that 
matter), and that it should therefore stop using SPV confirmation and switch to 
strict fullnode rejection of blocks that indicate a "next blocksize" that is 
too large or too small according to your equation?  OR will it simply follow 
the longest chain always, in which case, it trusts miners to be honest about 
how loaded (or unloaded) the transaction pool is?

---

As a general rule, consensus rules should restrict themselves to:

1.  The characteristics of the block.
2.  The characteristics of the transactions within the block.

The transaction pool is specifically those transaction that are NOT in any 
block, and 

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Good morning ZmnSCPxj, it must be where you are,


I suppose that we are each missing each other's point some.


I understand that nodes would not be expected to agree on the transaction pool 
and do not propose validating that the correct transactions are included in a 
block. I speak of probability and likelihood of a transaction being included in 
a block, implying a random element. I do not propose rejecting blocks on the 
basis that the next block size is stated too large or too small for the 
transaction pool, only that the block received conforms to the next block size 
given on the previous block. Yes, it could be cheated. Also, various nodes may 
have at times wildly different amounts of transactions waiting in the 
transaction pool compared to each other and there could be a great disparity 
between them. It would not be possible in any case I can think of to validate 
the next block size is correct for the current transaction pool. Even as it is 
now, nodes may include transactions in a block that no other nodes have even 
heard of, nodes have no way to validate that either. If the block is built on 
sufficiently, it is the blockchain.


I will post back the revised proposal to the list. I have fleshed parts of it 
out more, given more explanation and, tried this time not to recycle 
terminology.


Regards,

Damian Williamson


From: ZmnSCPxj 
Sent: Thursday, 7 December 2017 5:46:08 PM
To: Damian Williamson
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For 
Ordering Transactions In Blocks

Good morning Damian,

>As I understand it, each node would be aware independently of x transactions 
>waiting for confirmation, the transaction pool.

Each long-running node would have a view that is roughly the same as the view 
of every other long-running node.

However, suppose a node, Sleeping Beauty, was temporarily stopped for a day 
(for various reasons) then is started again.  That node cannot verify what the 
"consensus" transaction pool was during the time it was stopped -- it has no 
view of that.  It can only trust that the longest chain is valid -- but that 
means it is SPV for this particular rule.

>If next blocksize is broadcast with the completed block it would be a simple 
>matter to back confirm that.

It would not. Suppose Sleeping Beauty slept at block height 500,000.  On 
awakening, some node provides some purported block at height 500,001.  This 
block indicates some "next blocksize" for the block at height 500,002.  How 
does Sleeping Beauty know that the transaction pool at block 500,001 was of the 
correct size to provide the given "next blocksize"?  The only way, would be to 
look if there is some other chain with greater height that includes or does not 
include that block: that is, SPV confirmation.

How does Sleeping Beauty know it has caught up, and that its transaction pool 
is similar to that of its neighbors (who might be lying to it, for that 
matter), and that it should therefore stop using SPV confirmation and switch to 
strict fullnode rejection of blocks that indicate a "next blocksize" that is 
too large or too small according to your equation?  OR will it simply follow 
the longest chain always, in which case, it trusts miners to be honest about 
how loaded (or unloaded) the transaction pool is?

---

As a general rule, consensus rules should restrict themselves to:

1.  The characteristics of the block.
2.  The characteristics of the transactions within the block.

The transaction pool is specifically those transaction that are NOT in any 
block, and thus, are not safe to depend on for any consensus rules.

Regards,
ZmnSCPxj

___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Hello Jim,


The variable block sizes would not, as I understand it, be easily implemented 
by a solo miner.


You are right, there is presently nothing stopping a miner from ordering the 
transactions included by a priority that is not entirely based on the fee.


It may be possible to verify blocks conform to the proposal by showing that the 
probability for all transactions included in the block statistically conform to 
a probability distribution curve, if that is necessary and,  *if* the 
individual transaction priority can be recreated. I am not that deep into the 
mathematics, however, it may also be possible to use a similar method to do 
this just based on the fee, that statistically, the blocks conform to a fee 
distribution. Needs a clever mathematician.


It is certainly possible to verify that blocks conform to the expected size.


Honour is why people follow policy without enforcement. I may be in the wrong 
group. (sic)


Regards,

Damian Williamson


From: bitcoin-dev-boun...@lists.linuxfoundation.org 
<bitcoin-dev-boun...@lists.linuxfoundation.org> on behalf of Jim Renkel via 
bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Wednesday, 6 December 2017 4:18:11 PM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For 
Ordering Transactions In Blocks


As i understand it, the transactions to be included in a block are entirely up 
to the miner of that block.


What prevents a miner from implementing the proposal on their own?


If this is adopted as some kind of "policy", what forces a miner to follow it?

Jim Renkel


On 12/2/2017 10:07 PM, Damian Williamson via bitcoin-dev wrote:

# BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In 
Blocks

I admit, with my limited experience in the operation of the protocol, the 
section entitled 'Solution operation' may not be entirely correct but you will 
get the idea. If I have it wrong, please correct it back to the list.


## The problem:


Everybody wants value. Miners want to maximize revenue from fees. Consumers 
want transaction reliability and, (we presume) low fees.

Current transaction bandwidth limit is a limiting factor for both.


## Solution summary:


Provide each transaction with a transaction weight, being a function of the fee 
paid (on a curve), and the time waiting in the transaction pool (also on a 
curve) out to n days (n=30 ?); the transaction weight serving as the likelihood 
of a transaction being included in the current block, and then use a target 
block size.

Protocol enforcement to prevent high or low blocksize cheating to be handled by 
having the protocol determine the target size for the current block using; 
current transaction pool size x ( 1 / (144 x n days ) ) = transactions to be 
included in the current block.

The curves used for the weight of transactions would have to be appropriate.


## Pros:

* Maximizes transaction reliability.
* Maximizes possibility for consumer and business uptake.
* Maximizes total fees paid per block without reducing reliability; because of 
reliability, confidence and uptake are greater; therefore, more transactions 
and more transactions total at priority fees.
* Market determines fee paid for transaction priority.

* Fee recommendations work all the way out to 30 days or greater.

* Provides additional block entropy and greater security since there is less 
probability of predicting the next block.


## Cons:

* ?
* Must be first be programmed.
* Anything else?


## Solution operation:


As I have said, my simplistic view of the operation. If I have this wrong, 
please correct it back to the list.

1. The protocol determines the target block size.

2. Assign each transaction in the pool a transaction weight based on fee and 
time waiting in the transaction pool.

3. Begin selecting transactions to include in the current block using 
transaction weight as the likelihood of inclusion until target block size is 
met.

4. Solve block.


Regards,

Damian Williamson



___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

Good afternoon ZmnSCPxj,


I have posted some discussion on the need for this proposal and, some 
refinements to the proposal explanation (not changes to the intended operation) 
to the bitcoin-discuss list. I didn't exactly mean to double post but thought 
it could use the discussion and, not to post it again, I will link to it when 
(if) it turns up, or will post it back here as an update on request. Currently, 
that post is awaiting moderator approval. I have also rewritten the solution 
operation section a bit in that post, not the idea that is being conveyed. I 
have added an additional step, reject blocks that do not meet the target block 
size for the current block.

I suggest it still should be added to the solution operation, to broadcast the 
next target block size with the current block when it is solved. Using that 
method may answer a part of your concern.

As I understand it, each node would be aware independently of x transactions 
waiting for confirmation, the transaction pool. Each node would no doubt have 
its own idea about how many waiting transactions there are and which particular 
transactions exist. I do not see why each node could not just work with the 
information at hand, it is my understanding that is what happens now, even with 
solved blocks vs the longest chain. I have not followed why you foresee from my 
proposal the need for fullnodes to back confirm the previous blocks in that 
manner.

If next blocksize is broadcast with the completed block it would be a simple 
matter to back confirm that. With transaction weight (transaction priority) I 
am suggesting that value gives the likelihood of a transaction being included, 
presuming an element of randomness as to whether any particular transaction is 
then included or not. Back confirmations on a transaction basis would be 
impossible anyway, all that could be confirmed is that a particular block has 
transactions that conform to a probability curve, if the variables are known, 
fee amount and time waiting in the pool, then the transaction priority can be 
recreated to establish that the probability of a particular block conforms. I 
certainly do not foresee including the full transaction pool in each block.

I am also presuming blocksize as a number of transactions rather than KB.

My suggestion, if adopted, is to directly make the operation of transaction 
priority a part of the operational standard - even without verification that it 
is being followed. The result of full transactional reliability is actually in 
the interests of miners as much as anyone.

The benefit of the proposal, not directly stated, is variable sized blocks 
(uncapped block size).

Yes, I have learned not to recycle terminology. My apologies, I had not been 
made aware that terminology already had use. Perhaps it would be simpler to 
call the proposal that I am putting forward here Transaction Priority.

Regards,
Damian Williamson



From: ZmnSCPxj 
Sent: Wednesday, 6 December 2017 4:46:45 PM
To: Damian Williamson
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For 
Ordering Transactions In Blocks

Good morning Damian,

The primary problem in your proposal, as I understand it, is that the 
"transaction pool" is never committed to and is not part of consensus 
currently.  It is unlikely that the transaction pool will ever be part of 
consensus, as putting the transaction pool into consensus is effectively 
lifting the block size to include the entire transaction pool into each block.  
The issue is that the transaction pool is transient and future fullnodes cannot 
find the contents of the transaction pool at the time blocks were made and 
cannot verify the correctness of historical blocks.  Also, fullnodes using 
-blocksonly mode have no transaction pool and cannot verify incoming blocks for 
these new rules.

Applying a patch that follows this policy into Bitcoin Core without enforcing 
it in all fullnodes will simply lead to miners removing the patch in software 
they run, making it an exercise in futility to write, review, and test this 
code in the first place.

In addition, you reuse the term "weight" for something different than its 
current use.  Current use, is that the "weight" of a transaction, is the 
computed weight from the SegWit weight equation, measured in virtual units 
called "sipa", using the equation (4sipa / non-witness byte + 1sipa/witness 
byte).

Regards,
ZmnSCPxj




Sent with ProtonMail Secure Email.

 Original Message 
Subject: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For 
Ordering Transactions In Blocks
Local Time: December 6, 2017 3:38 AM
UTC Time: December 5, 2017 7:38 PM
From: bitcoin-dev@lists.linuxfoundation.org
To: bitcoin-dev@lists.linuxfoundation.org 




# BIP Proposal: UTWFOTIB - Use 

[bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson via bitcoin-dev]

# BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In 
Blocks

I admit, with my limited experience in the operation of the protocol, the 
section entitled 'Solution operation' may not be entirely correct but you will 
get the idea. If I have it wrong, please correct it back to the list.


## The problem:


Everybody wants value. Miners want to maximize revenue from fees. Consumers 
want transaction reliability and, (we presume) low fees.

Current transaction bandwidth limit is a limiting factor for both.


## Solution summary:


Provide each transaction with a transaction weight, being a function of the fee 
paid (on a curve), and the time waiting in the transaction pool (also on a 
curve) out to n days (n=30 ?); the transaction weight serving as the likelihood 
of a transaction being included in the current block, and then use a target 
block size.

Protocol enforcement to prevent high or low blocksize cheating to be handled by 
having the protocol determine the target size for the current block using; 
current transaction pool size x ( 1 / (144 x n days ) ) = transactions to be 
included in the current block.

The curves used for the weight of transactions would have to be appropriate.


## Pros:

* Maximizes transaction reliability.
* Maximizes possibility for consumer and business uptake.
* Maximizes total fees paid per block without reducing reliability; because of 
reliability, confidence and uptake are greater; therefore, more transactions 
and more transactions total at priority fees.
* Market determines fee paid for transaction priority.

* Fee recommendations work all the way out to 30 days or greater.

* Provides additional block entropy and greater security since there is less 
probability of predicting the next block.


## Cons:

* ?
* Must be first be programmed.
* Anything else?


## Solution operation:


As I have said, my simplistic view of the operation. If I have this wrong, 
please correct it back to the list.

1. The protocol determines the target block size.

2. Assign each transaction in the pool a transaction weight based on fee and 
time waiting in the transaction pool.

3. Begin selecting transactions to include in the current block using 
transaction weight as the likelihood of inclusion until target block size is 
met.

4. Solve block.


Regards,

Damian Williamson
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev