Comments: 1) cblock seems a reasonable way to extend the protocol. Further wrapping should probably be done at the stream level.
2) zlib has crappy security track record. 3) A fallback path to non-compressed is required, should compression fail or crash. 4) Most blocks and transactions have runs of zeroes and/or highly common bit-patterns, which contributes to useful compression even at smaller sizes. Peter Ts's most recent numbers bear this out. zlib has a dictionary (32K?) which works well with repeated patterns such as those you see with concatenated runs of transactions. 5) LZO should provide much better compression, at a cost of CPU performance and using a less-reviewed, less-field-tested library. On Tue, Nov 10, 2015 at 11:30 AM, Tier Nolan via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > > > On Tue, Nov 10, 2015 at 4:11 PM, Peter Tschipper < > peter.tschip...@gmail.com> wrote: > >> There are better ways of sending new blocks, that's certainly true but >> for sending historical blocks and seding transactions I don't think so. >> This PR is really designed to save bandwidth and not intended to be a huge >> performance improvement in terms of time spent sending. >> > > If the main point is for historical data, then sticking to just blocks is > the best plan. > > Since small blocks don't compress well, you could define a "cblocks" > message that handles multiple blocks (just concatenate the block messages > as payload before compression). > > The sending peer could combine blocks so that each cblock is compressing > at least 10kB of block data (or whatever is optimal). It is probably worth > specifying a maximum size for network buffer reasons (either 1MB or 1 block > maximum). > > Similarly, transactions could be combined together and compressed "ctxs". > The inv messages could be modified so that you can request groups of 10-20 > transactions. That would depend on how much of an improvement compressed > transactions would represent. > > More generally, you could define a message which is a compressed message > holder. That is probably to complex to be worth the effort though. > > > >> >> On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via bitcoin-dev < >> <bitcoin-dev@lists.linuxfoundation.org> >> bitcoin-dev@lists.linuxfoundation.org> wrote: >> >>> On Mon, Nov 9, 2015 at 5:58 PM, gladoscc via bitcoin-dev < >>> bitcoin-dev@lists.linuxfoundation.org> wrote: >>> >>> >>>> I think 25% bandwidth savings is certainly considerable, especially for >>>> people running full nodes in countries like Australia where internet >>>> bandwidth is lower and there are data caps. >>>> >>> >>> This reinforces the idea that such trade-off decisions should be be >>> local and negotiated between peers, not a required feature of the network >>> P2P. >>> >>> >>> -- >>> Johnathan Corgan >>> Corgan Labs - SDR Training and Development Services >>> http://corganlabs.com >>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >>> >> >> >> _______________________________________________ >> bitcoin-dev mailing >> listbitcoin-dev@lists.linuxfoundation.orghttps://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> >> > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >
_______________________________________________ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
