Hi pukkamustard, The rationale was/is using a power of two while also staing below the ~63k block size of GNUnet CORE (so we can fit into one message), which itself arises from the use of a 16-bit integer for message size, which itself was used (a) to fit into a single IP packet, (b) to fundamentally avoid problems with attackers sending us individual messages that don't fit into the heap on smaller devices, (c) to reduce potential impacts from (bad code failing to properly guard against) integer arithmetic overflows (as the 16-bit inputs don't overflow so easily after they are promoted to (usually) 32-bit 'int's), and (d) to limit the impact of malformed blocks by allowing ECRS to detect bogus blocks at a fine granularity level. Also, it was a massive increase from the _original_ (2001) design, where we used 1k blocks to fit into unfragmented UDP packets on Ethernet ;-).
Note that _today_, if I had time to re-implement FS, I would probably stick to this block size in terms of verification / CORE-transmission, but would not issue queries for individual blocks. Instead, it would probably be better to issue a query for a 'stream' of blocks and require peers to store all blocks for a file together. That would likely result in a better privacy/availability trade-off than what we have today. Happy hacking! Christian On 5/13/20 4:26 PM, pukkamustard wrote: > Hello GNUnet, > > I am interested in the reasons for using a block size of 32kB in the > file-sharing application [1]. > > In the paper "Efficient Sharing of Encrypted Data (2002)" [2] arguments > are made for using a block size of 1kB: > >> One could argue that a size of 1k is far too small. The rationale >> behind thisfile size is that UDP, the transport mechanism used >> bygnunet has (on Ethernet)an optimal packet size that is slightly >> above 1k. Asgnunet must add headers tothe packets, the size >> approximates this number. Furthermore, many file systemscan be >> formatted with a block-size of 1k. > > In "An Encoding for Censorship-Resistant Sharing (2003)" the block size > 32kB is used without giving concrete arguments (or I missed them). > > Is there a rationale for using 32kB? What are the experiences whit using > this block size? Would you use a different size today? > > I initially posted this question on IRC. Thanks to @tg and @dvn for the > input and pointing me here. > > -pukkamustard > > > [1] https://git.gnunet.org/gnunet.git/tree/src/fs/fs.h#n37 > [2] https://grothoff.org/christian/esed.pdf > [3] https://grothoff.org/christian/ecrs.pdf >
