Re: [bitcoin-dev] Compressed Bitcoin Transactions

2024-01-23 Thread Tom Briar via bitcoin-dev
Hi Jonas, As it turns out, most of our size savings come from eliminating unneeded hashes and public keys, which get recovered on decompression. gzip actually expands transactions due to the way it attempts to compress pseudorandom data, my numbers show a legacy transaction of 222 bytes being

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2024-01-18 Thread Jonas Schnelli via bitcoin-dev
One point to add here is that, while V1 non-encrypted p2p traffic could be compressed on a different OSI layer in theory, v2 encrypted traffic – due to its pseudorandom nature – will likely have no size savings and thus need to be compressed on the application layer with a proposal like this.

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2024-01-16 Thread Tom Briar via bitcoin-dev
Hi, In addition to the use cases listed in the schema, such as steganography, satellite, and radio broadcast, an application can be made for Peer-to-peer communication between Bitcoin nodes. Except when compressing the Txid/Vout, which is optional, Transactions can gain up to 30% size savings

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2024-01-11 Thread Tom Briar via bitcoin-dev
Hi, After reviewing all the feedback and writing a reference implementation, I have linked the updated schema and a Draft PR for a reference Implementation to Bitcoin Core. Some of the major changes consist of: Removing the grinding of the nLocktime in favor of a relative block height,

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2024-01-05 Thread Andrew Poelstra via bitcoin-dev
Thanks Tom. It looks like you posted a text-scrape of the rendered markdown, which is hard to read. For posterity here is the full text. Best Andrew === begin compressed_transactions.md === # Compressed Transaction Schema By (Tom Briar) and (Andrew Poelstra) ## 1. Abstract With this

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2024-01-05 Thread Tom Briar via bitcoin-dev
Hi, After reviewing all the feedback and writing a reference implementation, I have linked the updated schema and a Draft PR for a reference Implementation to Bitcoin Core. Some of the major changes consist of: - Removing the grinding of the nLocktime in favor of a relative block height,

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2023-09-05 Thread Tom Briar via bitcoin-dev
Hi Peter, Currently, if we’re given a lock time that is non zero, we drop the 16 most significant bits and grind through until we have a valid signature. Therefore I am hesitant to add more fields to grind through, because it can get out of hand in decompression time really quickly. That said

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2023-09-05 Thread Peter Todd via bitcoin-dev
On Fri, Sep 01, 2023 at 01:56:18PM +, Andrew Poelstra via bitcoin-dev wrote: > We can swag what the space savings would be: there are 122MM utxos right > now, which is a bit under 2^27. So assuming a uniform distribution of > prefixes we'd need to specify 28 bits to identify a UTXO. To

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2023-09-01 Thread Tom Briar via bitcoin-dev
Hi Jonas, I’m working to get numbers based on both historical data and from fuzz tests but I’m in the middle of updating the code to match the doc, I should have it finished before the end of the week. We estimate that 100 blocks is safe from reorg, that is the same policyfor spending coin

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2023-09-01 Thread Jonas Schnelli via bitcoin-dev
Hi Tom > I've been working on a way to compress bitcoin transactions for transmission > through steganography, satellite broadcasting, Interesting. Some size numbers (vs plain, vs gzip) would be nice. Maybe add a definition to your BIP that makes clear when NOT to use height/index due to

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2023-09-01 Thread Tom Briar via bitcoin-dev
Hi Fabian, Yes as Andrew said, creating a prefix tree is going to take up more space then simply the block height and then an index for the UTXO in the block. We removed the vout from the encoding by doing almost exactly what you said per block where it’s a flattened index over all the

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2023-09-01 Thread Andrew Poelstra via bitcoin-dev
Hi Fabian, We did consider indexing all txos -- even, amusingly, by using ordinals -- but decided that the extra index requirements for the decompressor (which otherwise just requires a bit of extra CPU cycles but nothing beyond a normal Core node). A while ago we looked into putting the whole

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2023-09-01 Thread Fabian via bitcoin-dev
Hi Tom, I realized I simplified my message a bit too much. Of course an index of the UTXO set would also need a height, so I rather meant some kind of composite reference I guess. An index alone might be enough if a height has been pre-agreed which could still be compatible with the use-cases

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2023-09-01 Thread Fabian via bitcoin-dev
Hi Tom, without having gone into the details yet, thanks for the great effort you have put into this research and implementation already! > The bulk of our size savings come from replacing the prevout of each input by > a block height and index. Have you also considered using just an index

Re: [bitcoin-dev] Compressed Bitcoin Transactions

2023-08-31 Thread Andrew Poelstra via bitcoin-dev
On Thu, Aug 31, 2023 at 09:30:16PM +, Tom Briar via bitcoin-dev wrote: > Hey everyone, > > I've been working on a way to compress bitcoin transactions for transmission > throughsteganography, satellite broadcasting, > and other low bandwidth channels with high CPU availability on

[bitcoin-dev] Compressed Bitcoin Transactions

2023-08-31 Thread Tom Briar via bitcoin-dev
Hey everyone, I've been working on a way to compress bitcoin transactions for transmission throughsteganography, satellite broadcasting, and other low bandwidth channels with high CPU availability on decompression.