Hello everyone, as discussed in a different mail exchange[1], the BIO API isn't considered a great API and it was suggested to improve on it. The attached patch is an attempt to both improve the API and add memory-backed buffers as proposed in the above-mentioned discussion.
Please note: this patch is not complete; it doesn't have tests (though they were made to check that the API works) and most importantly doesn't have modifications to port the rest of the code to the new API. This is because this patch is only for review, so I will take any feedback, make changes if needed and when we agree on a certain version I will also change all the callers to BIO and provide the final patch. The patch adds I/O on already-allocated buffers and provides a new API similar to the one used for databases, as suggested by Christian. It doesn't add I/O for arrays as I wasn't sure how to implement it, though I had made an attempt before yanking it out and some hints at it can be seen here and there (though they don't affect the rest of the code.) To clarify here one of the design choices, the "spec" API uses the handle in the spec definition instead of in the "commit" function so as to declare all the operations on multiple sources/destinations all together and execute them all in one go. I think it's fairly convenient to do it this way instead of creating multiple specs, but this is just my opinion and is not backed by any objective evaluation :) Thanks, A.V. --- [1] https://lists.gnu.org/archive/html/gnunet-developers/2020-04/msg00015.html
>From 0cf140eae8a36699ef5cb917e1c60f1b3b020c0f Mon Sep 17 00:00:00 2001 From: Alessio Vanni <[email protected]> Date: Sat, 9 May 2020 14:20:48 +0200 Subject: [PATCH] New BIO API --- src/include/gnunet_bio_lib.h | 550 ++++++++++++++--- src/util/bio.c | 1132 ++++++++++++++++++++++++++++------ 2 files changed, 1397 insertions(+), 285 deletions(-) diff --git a/src/include/gnunet_bio_lib.h b/src/include/gnunet_bio_lib.h index 2f715ec97..2c9048a2f 100644 --- a/src/include/gnunet_bio_lib.h +++ b/src/include/gnunet_bio_lib.h @@ -42,6 +42,9 @@ extern "C" #endif #endif +/****************************** READING API ********************************/ + + /** * Handle for buffered reading. */ @@ -59,71 +62,75 @@ GNUNET_BIO_read_open (const char *fn); /** - * Close an open file. Reports if any errors reading - * from the file were encountered. + * Create a handle from an existing buffer. * - * @param h file handle - * @param emsg set to the error message - * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise + * @param buffer the buffer to use + * @param size the size in bytes of the buffer + (can be smaller than the real allocated size) + * @return IO handle on success, NULL on error */ -int -GNUNET_BIO_read_close (struct GNUNET_BIO_ReadHandle *h, char **emsg); +struct GNUNET_BIO_ReadHandle * +GNUNET_BIO_read_create (void *buffer, size_t size); /** - * Read the contents of a binary file into a buffer. + * Destroy an IO handle. If the handle was using a file, the file will be + * closed. If the handle was using a buffer allocated externally, the buffer + * will @b not be freed. * - * @param h handle to an open file - * @param what describes what is being read (for error message creation) - * @param result the buffer to write the result to - * @param len the number of bytes to read - * @return #GNUNET_OK on success, #GNUNET_SYSERR on failure + * @param h the handle + * @param emsg set to the (allocated) error message + * if emsg is present, the return value is #GNUNET_SYSERR + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_read (struct GNUNET_BIO_ReadHandle *h, const char *what, - void *result, size_t len); +GNUNET_BIO_read_destroy (struct GNUNET_BIO_ReadHandle *h, char **emsg); /** - * Read the contents of a binary file into a buffer. + * Read some contents into a buffer. * - * @param h handle to an open file - * @param file name of the source file - * @param line line number in the source file + * @param h the IO handle to read from + * @param what describes what is being read (for error message creation) * @param result the buffer to write the result to * @param len the number of bytes to read * @return #GNUNET_OK on success, #GNUNET_SYSERR on failure */ int -GNUNET_BIO_read_fn (struct GNUNET_BIO_ReadHandle *h, - const char *file, int line, - void *result, size_t len); +GNUNET_BIO_read (struct GNUNET_BIO_ReadHandle *h, + const char *what, + void *result, + size_t len); + /** - * Read 0-terminated string from a file. + * Read a 0-terminated string from a handle. * - * @param h handle to an open file + * @param h the IO handle to read from * @param what describes what is being read (for error message creation) - * @param result the buffer to store a pointer to the (allocated) string to - * (note that *result could be set to NULL as well) + * @param result the buffer to store a pointer to the (allocated) string + * can be NULL in certain cases * @param max_length maximum allowed length for the string - * @return #GNUNET_OK on success, #GNUNET_SYSERR on failure + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_read_string (struct GNUNET_BIO_ReadHandle *h, const char *what, - char **result, size_t max_length); +GNUNET_BIO_read_string (struct GNUNET_BIO_ReadHandle *h, + const char *what, + char **result, + size_t max_length); /** - * Read metadata container from a file. + * Read a metadata container from a handle. * - * @param h handle to an open file + * @param h the IO handle to read from * @param what describes what is being read (for error message creation) - * @param result the buffer to store a pointer to the (allocated) metadata + * @param result the buffer to store a pointer to the (allocated) metadata. * @return #GNUNET_OK on success, #GNUNET_SYSERR on failure */ int -GNUNET_BIO_read_meta_data (struct GNUNET_BIO_ReadHandle *h, const char *what, +GNUNET_BIO_read_meta_data (struct GNUNET_BIO_ReadHandle *h, + const char *what, struct GNUNET_CONTAINER_MetaData **result); @@ -131,70 +138,57 @@ GNUNET_BIO_read_meta_data (struct GNUNET_BIO_ReadHandle *h, const char *what, * Read a float. * * @param h hande to open file + * @param what what is being read (for error message creation) * @param f address of float to read */ -#define GNUNET_BIO_read_float(h, f) (GNUNET_BIO_read_fn (h, __FILE__, __LINE__, \ - f, sizeof(float))) +#define GNUNET_BIO_read_float(h, what, f) (GNUNET_BIO_read (h, \ + what, \ + f, \ + sizeof(float))) /** * Read a double. * * @param h hande to open file + * @param what what is being read (for error message creation) * @param f address of double to read */ -#define GNUNET_BIO_read_double(h, f) (GNUNET_BIO_read_fn (h, __FILE__, __LINE__, \ - f, sizeof(double))) +#define GNUNET_BIO_read_double(h, what, f) (GNUNET_BIO_read (h, \ + what, \ + f, \ + sizeof(double))) /** * Read an (u)int32_t. * - * @param h hande to open file - * @param file name of the source file - * @param line line number in the code - * @param i address of 32-bit integer to read + * @param h the IO handle to read from + * @param what describes what is being read (for error message creation) + * @param i where to store the data * @return #GNUNET_OK on success, #GNUNET_SYSERR on error */ int -GNUNET_BIO_read_int32__ (struct GNUNET_BIO_ReadHandle *h, const char *file, - int line, int32_t *i); - - -/** - * Read an (u)int32_t. - * - * @param h hande to open file - * @param i address of 32-bit integer to read - */ -#define GNUNET_BIO_read_int32(h, i) GNUNET_BIO_read_int32__ (h, __FILE__, \ - __LINE__, \ - (int32_t *) i) +GNUNET_BIO_read_int32 (struct GNUNET_BIO_ReadHandle *h, + const char *what, + int32_t *i); /** * Read an (u)int64_t. * - * @param h hande to open file - * @param file name of the source file - * @param line line number in the code - * @param i address of 64-bit integer to read + * @param h the IO handle to read from + * @param what describes what is being read (for error message creation) + * @param i where to store the data * @return #GNUNET_OK on success, #GNUNET_SYSERR on error */ int -GNUNET_BIO_read_int64__ (struct GNUNET_BIO_ReadHandle *h, const char *file, - int line, int64_t *i); +GNUNET_BIO_read_int64 (struct GNUNET_BIO_ReadHandle *h, + const char *what, + int64_t *i); -/** - * Read an (u)int64_t. - * - * @param h hande to open file - * @param i address of 64-bit integer to read - */ -#define GNUNET_BIO_read_int64(h, i) GNUNET_BIO_read_int64__ (h, __FILE__, \ - __LINE__, \ - (int64_t *) i) +/****************************** WRITING API ****************************/ /** @@ -202,6 +196,7 @@ GNUNET_BIO_read_int64__ (struct GNUNET_BIO_ReadHandle *h, const char *file, */ struct GNUNET_BIO_WriteHandle; + /** * Open a file for writing. * @@ -213,30 +208,52 @@ GNUNET_BIO_write_open (const char *fn); /** - * Close an open file for writing. + * Create a handle from an existing buffer. + * + * @param buffer the buffer to use + * @param size the size in bytes of the buffer + (can be smaller than the real allocated size) + * @return IO handle on success, NULL on error + */ +struct GNUNET_BIO_WriteHandle * +GNUNET_BIO_write_create (void *buffer, size_t size); + + +/** + * Destroy an IO handle. If the handle was using a file, the file will be + * closed. If the handle was using a buffer allocated externally, the buffer + * will @b not be freed. * - * @param h file handle - * @return GNUNET_OK on success, GNUNET_SYSERR otherwise + * @param h the handle + * @param emsg set to the (allocated) error message + * if emsg is present, the return value is #GNUNET_SYSERR + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_write_close (struct GNUNET_BIO_WriteHandle *h); +GNUNET_BIO_write_destroy (struct GNUNET_BIO_WriteHandle *h, char **emsg); /** - * Write a buffer to a file. + * Write a buffer to an IO handle. + * The destination depends on how the handle was created. * - * @param h handle to open file + * @param h the IO handle to write to + * @param what what is being written (for error message creation) * @param buffer the data to write * @param n number of bytes to write - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_write (struct GNUNET_BIO_WriteHandle *h, const void *buffer, +GNUNET_BIO_write (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const void *buffer, size_t n); /** - * Force a buffered writer to flush its buffer + * Force a file-based buffered writer to flush its buffer. + * If the handle does not use a file, this function simply returns #GNUNET_OK + * without doing anything. * * @param h the writer handle * @return #GNUNET_OK upon success. Upon failure #GNUNET_SYSERR is returned and @@ -247,25 +264,30 @@ GNUNET_BIO_flush (struct GNUNET_BIO_WriteHandle *h); /** - * Write a string to a file. + * Write a string to an IO handle. * - * @param h handle to open file - * @param s string to write (can be NULL) - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @param h the IO handle + * @param what what is being written (for error message creation) + * @param s 0-terminated string to write (can be NULL) + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_write_string (struct GNUNET_BIO_WriteHandle *h, const char *s); +GNUNET_BIO_write_string (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const char *s); /** - * Write metadata container to a file. + * Write metadata container to a IO handle. * - * @param h handle to open file + * @param h the IO handle + * @param what what is being written (for error message creation) * @param m metadata to write - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int GNUNET_BIO_write_meta_data (struct GNUNET_BIO_WriteHandle *h, + const char *what, const struct GNUNET_CONTAINER_MetaData *m); @@ -273,40 +295,374 @@ GNUNET_BIO_write_meta_data (struct GNUNET_BIO_WriteHandle *h, * Write a float. * * @param h hande to open file + * @param what what is being written (for error message creation) * @param f float to write (must be a variable) */ -#define GNUNET_BIO_write_float(h, f) GNUNET_BIO_write (h, &f, sizeof(float)) +#define GNUNET_BIO_write_float(h, what, f) GNUNET_BIO_write (h, \ + what, \ + &f, \ + sizeof(float)) /** * Write a double. * * @param h hande to open file + * @param what what is being written (for error message creation) * @param f double to write (must be a variable) */ -#define GNUNET_BIO_write_double(h, f) GNUNET_BIO_write (h, &f, sizeof(double)) +#define GNUNET_BIO_write_double(h, what, f) GNUNET_BIO_write (h, \ + what, \ + &f, \ + sizeof(double)) /** - * Write an (u)int32_t. + * Write a (u)int32 to an IO handle. * - * @param h hande to open file - * @param i 32-bit integer to write - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @param h the IO handle + * @param what what is being written (for error message creation) + * @param i 32-big integer to write + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_write_int32 (struct GNUNET_BIO_WriteHandle *h, int32_t i); +GNUNET_BIO_write_int32 (struct GNUNET_BIO_WriteHandle *h, + const char *what, + int32_t i); /** - * Write an (u)int64_t. + * Write a (u)int64 to an IO handle. * - * @param h hande to open file + * @param h the IO handle + * @param what what is being written (for error message creation) * @param i 64-bit integer to write - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise + */ +int +GNUNET_BIO_write_int64 (struct GNUNET_BIO_WriteHandle *h, + const char *what, + int64_t i); + + +/*************************** READ/WRITE SPEC API *************************/ + + +/** + * These values identify a "primitive type" for data to be read/written. + */ +#define GNUNET_BIO_SPEC_OBJECT 0 +#define GNUNET_BIO_SPEC_INT32 2 +#define GNUNET_BIO_SPEC_INT64 4 +#define GNUNET_BIO_SPEC_FLOAT 8 +#define GNUNET_BIO_SPEC_DOUBLE 16 +#define GNUNET_BIO_SPEC_STRING 32 +#define GNUNET_BIO_SPEC_META 64 + + +/********************************* READ SPEC API ************************/ + + +/** + * Description of a read operation. + */ +struct GNUNET_BIO_ReadSpec +{ + /** + * What is being read in this operation. + */ + const char *what; + + /** + * The "type" of the data being read. + */ + int type; + + /** + * The handle to read from. + */ + struct GNUNET_BIO_ReadHandle *h; + + /** + * How many bytes should be read from the handle. + */ + size_t size; + + /** + * Where to store the data. + */ + void *result; +}; + + +/** + * Create a read spec to read a certain amount of bytes. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @param size how many bytes should be read + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_object (struct GNUNET_BIO_ReadHandle *h, + const char *what, + void *result, + size_t size); + + +/** + * Create a read spec to read an (u)int32. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_int32 (struct GNUNET_BIO_ReadHandle *h, + const char *what, + int32_t *result); + + +/** + * Create a read spec to read an (u)int64. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_int64 (struct GNUNET_BIO_ReadHandle *h, + const char *what, + int64_t *result); + + +/** + * Create a read spec to read a float. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_float (struct GNUNET_BIO_ReadHandle *h, + const char *what, + float *result); + + +/** + * Create a read spec to read a double. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_double (struct GNUNET_BIO_ReadHandle *h, + const char *what, + double *result); + + +/** + * Create a read spec to read a 0-terminated string. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store a pointer to the (allocated) string + * can be NULL in certain cases + * @param max_length maximum allowed length for the string + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_string (struct GNUNET_BIO_ReadHandle *h, + const char *what, + char **result, + size_t max_length); + + +/** + * Create a read spec to read a metadata container. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result the buffer to store a pointer to the (allocated) metadata. + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_meta_data (struct GNUNET_BIO_ReadHandle *h, + const char *what, + struct GNUNET_CONTAINER_MetaData **result); + + +/** + * End of read operations specification. + */ +#define GNUNET_BIO_read_spec_end() { NULL, 0, NULL, 0, NULL } + + +/** + * Execute the provided read operations. + * + * @param rs an array of read spec + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise + */ +int +GNUNET_BIO_read_spec_commit (struct GNUNET_BIO_ReadSpec *rs); + + +/******************************* WRITE SPEC API ****************************/ + + +/** + * Description of a write operation. + */ +struct GNUNET_BIO_WriteSpec +{ + /** + * What is being read in this operation. + */ + const char *what; + + /** + * The "type" of the data being written. + */ + int type; + + /** + * The handle to write to. + */ + struct GNUNET_BIO_WriteHandle *h; + + /** + * How many bytes should be written to the handle. + */ + size_t size; + + /** + * The data to write to the handle. + */ + const void *source; +}; + + +/** + * Create a write spec to write a certain amount of bytes. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source the data to write to the handle + * @param size how many bytes should be written + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_object (struct GNUNET_BIO_WriteHandle *h, + const char *what, + void *source, + size_t size); + + +/** + * Create a write spec to write an (u)int32. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source a pointer to a 32-bit integer + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_int32 (struct GNUNET_BIO_WriteHandle *h, + const char *what, + int32_t *source); + + +/** + * Create a write spec to write an (u)int64. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source a pointer to a 64-bit integer + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_int64 (struct GNUNET_BIO_WriteHandle *h, + const char *what, + int64_t *source); + + +/** + * Create a write spec to write a float. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source a pointer to a float + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_float (struct GNUNET_BIO_WriteHandle *h, + const char *what, + float *source); + + +/** + * Create a write spec to write a double. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source a pointer to a double + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_double (struct GNUNET_BIO_WriteHandle *h, + const char *what, + double *source); + + +/** + * Create a write spec to write a string + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source the 0-terminated string to write + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_string (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const char *source); + + +/** + * Create a write spec to write a metadata container + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source the metadata to write + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_meta_data (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const struct GNUNET_CONTAINER_MetaData *source); + + +/** + * End of write operations specification. + */ +#define GNUNET_BIO_write_spec_end() { NULL, 0, NULL, 0, NULL } + + +/** + * Execute the provided write operations. + * + * @param ws an array of write specs + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_write_int64 (struct GNUNET_BIO_WriteHandle *h, int64_t i); +GNUNET_BIO_write_spec_commit (struct GNUNET_BIO_WriteSpec *ws); #if 0 /* keep Emacsens' auto-indent happy */ diff --git a/src/util/bio.c b/src/util/bio.c index e05258f73..424573945 100644 --- a/src/util/bio.c +++ b/src/util/bio.c @@ -36,39 +36,64 @@ /** - * Size for I/O buffers. + * Size for file-based I/O buffers. */ #define BIO_BUFFER_SIZE 65536 /** - * Maximum size allowed for meta data written/read from disk. + * Maximum read/written size allowed for meta data. * File-sharing limits to 64k, so this should be rather generous. */ #define MAX_META_DATA (1024 * 1024) +/** + * Enum used internally to know how buffering is handled. + * + * The idea is that by using an enum, BIO can be extended to support other + * kind of "backend" for buffering (or even just for formatted I/O.) + */ +enum IOType +{ + /** + * The handle uses a file to read/write data. + */ + IO_FILE = 0, + + /** + * The data is entirely in memory withing a fixed-size buffer. + */ + IO_BUFFER, +}; + + /** * Handle for buffered reading. */ struct GNUNET_BIO_ReadHandle { /** - * Underlying file abstraction. + * The "backend" type. + */ + enum IOType type; + + /** + * Handle to a file on disk, if @e type is #IO_FILE. */ struct GNUNET_DISK_FileHandle *fd; /** - * Error message, NULL if there were no errors. + * Error message. NULL is there were no errors. */ char *emsg; /** - * I/O buffer. Allocated at the end of the struct, do not free! + * I/O buffer. Do @b not free! */ char *buffer; /** - * Number of bytes available in read @e buffer. + * Number of bytes available in @e buffer. */ size_t have; @@ -78,7 +103,7 @@ struct GNUNET_BIO_ReadHandle size_t size; /** - * Current read offset in @e buffer. + * Current offset in @buffer. */ off_t pos; }; @@ -87,7 +112,7 @@ struct GNUNET_BIO_ReadHandle /** * Open a file for reading. * - * @param fn file name to be opened + * @param fn name of file to open * @return IO handle on success, NULL on error */ struct GNUNET_BIO_ReadHandle * @@ -103,78 +128,111 @@ GNUNET_BIO_read_open (const char *fn) h->buffer = (char *) &h[1]; h->size = BIO_BUFFER_SIZE; h->fd = fd; + h->type = IO_FILE; + return h; +} + + +/** + * Create a handle from an existing buffer. + * + * @param buffer the buffer to use + * @param size the size in bytes of the buffer + (can be smaller than the real allocated size) + * @return IO handle on success, NULL on error + */ +struct GNUNET_BIO_ReadHandle * +GNUNET_BIO_read_create (void *buffer, size_t size) +{ + struct GNUNET_BIO_ReadHandle *h; + if (NULL == buffer || 0 == size) + return NULL; + h = GNUNET_new (struct GNUNET_BIO_ReadHandle); + h->buffer = buffer; + h->size = size; + h->type = IO_BUFFER; return h; } /** - * Close an open file. Reports if any errors reading - * from the file were encountered. + * Destroy an IO handle. If the handle was using a file, the file will be + * closed. If the handle was using a buffer allocated externally, the buffer + * will @b not be freed. * - * @param h file handle - * @param emsg set to the error message + * @param h the handle + * @param emsg set to the (allocated) error message + * if emsg is present, the return value is #GNUNET_SYSERR * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_read_close (struct GNUNET_BIO_ReadHandle *h, char **emsg) +GNUNET_BIO_read_destroy (struct GNUNET_BIO_ReadHandle *h, char **emsg) { int err; err = (NULL == h->emsg) ? GNUNET_OK : GNUNET_SYSERR; - if (emsg != NULL) + if (NULL != emsg) *emsg = h->emsg; else GNUNET_free_non_null (h->emsg); - GNUNET_DISK_file_close (h->fd); + switch (h->type) + { + case IO_FILE: + GNUNET_DISK_file_close (h->fd); + break; + case IO_BUFFER: + break; + default: + break; + } GNUNET_free (h); return err; } /** - * Read the contents of a binary file into a buffer. + * Internal function used to read from a file. + * The data read is first buffered so further reads will not generate + * disk activities unless necessary. * - * @param h handle to an open file + * @param h the IO handle * @param what describes what is being read (for error message creation) - * @param result the buffer to write the result to - * @param len the number of bytes to read - * @return #GNUNET_OK on success, #GNUNET_SYSERR on failure + * @param result where to write the data + * @param len how many bytes to read + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ -int -GNUNET_BIO_read (struct GNUNET_BIO_ReadHandle *h, - const char *what, - void *result, - size_t len) +static int +read_file (struct GNUNET_BIO_ReadHandle *h, + const char *what, + char *result, + size_t len) { - char *dst = result; size_t min; size_t pos; ssize_t ret; - if (NULL != h->emsg) - return GNUNET_SYSERR; pos = 0; do { - /* first, use buffer */ + /* Try to use the buffer first */ min = h->have - h->pos; - if (min > 0) + if (0 < min) { - if (min > len - pos) + if (len - pos < min) min = len - pos; - GNUNET_memcpy (&dst[pos], &h->buffer[h->pos], min); + GNUNET_memcpy (&result[pos], &h->buffer[h->pos], min); h->pos += min; pos += min; } if (pos == len) - return GNUNET_OK; /* done! */ + return GNUNET_OK; GNUNET_assert (((off_t) h->have) == h->pos); - /* fill buffer */ + /* Fill the buffer with more data */ ret = GNUNET_DISK_file_read (h->fd, h->buffer, h->size); if (-1 == ret) { GNUNET_asprintf (&h->emsg, - _ ("Error reading `%s': %s"), + _ ("Error while reading `%s' from file: %s"), what, strerror (errno)); return GNUNET_SYSERR; @@ -182,7 +240,7 @@ GNUNET_BIO_read (struct GNUNET_BIO_ReadHandle *h, if (0 == ret) { GNUNET_asprintf (&h->emsg, - _ ("Error reading `%s': %s"), + _ ("Error while reading `%s' from file: %s"), what, _ ("End of file")); return GNUNET_SYSERR; @@ -190,44 +248,132 @@ GNUNET_BIO_read (struct GNUNET_BIO_ReadHandle *h, h->pos = 0; h->have = ret; } - while (pos < len); /* should always be true */ + while (pos < len); + return GNUNET_OK; +} + + +/** + * Internal function to read from a buffer. + * The data is simply copied from the managed buffer to the destination + * buffer. Additionally, attempting to read too much data will result in an + * error instead of implicitly reading less data. + * + * @param h the IO handle + * @param what describes what is being read (for error message creation) + * @param result where to write the data + * @param len how many bytes to read + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise + */ +static int +read_buffer (struct GNUNET_BIO_ReadHandle *h, + const char *what, + char *result, + size_t len) +{ + if (0 == len) + return GNUNET_OK; + if (h->size < len || h->size - h->pos < len) + { + GNUNET_asprintf (&h->emsg, + _ ("Error while reading `%s' from buffer: %s"), + what, + _ ("Not enough data left")); + return GNUNET_SYSERR; + } + GNUNET_memcpy (result, h->buffer+h->pos, len); + h->pos += len; return GNUNET_OK; } /** - * Read the contents of a binary file into a buffer. + * Read some contents into a buffer. * - * @param h handle to an open file - * @param file name of the source file - * @param line line number in the source file + * @param h the IO handle to read from + * @param what describes what is being read (for error message creation) * @param result the buffer to write the result to * @param len the number of bytes to read * @return #GNUNET_OK on success, #GNUNET_SYSERR on failure */ int -GNUNET_BIO_read_fn (struct GNUNET_BIO_ReadHandle *h, - const char *file, - int line, - void *result, - size_t len) +GNUNET_BIO_read (struct GNUNET_BIO_ReadHandle *h, + const char *what, + void *result, + size_t len) +{ + char *dst = result; + int ret = GNUNET_OK; + + if (NULL != h->emsg) + return GNUNET_SYSERR; + + switch (h->type) + { + case IO_FILE: + ret = read_file (h, what, dst, len); + break; + case IO_BUFFER: + ret = read_buffer (h, what, dst, len); + break; + } + + return ret; +} + + +/** + * Read an (u)int32_t. + * + * @param h the IO handle to read from + * @param what describes what is being read (for error message creation) + * @param i where to store the data + * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + */ +int +GNUNET_BIO_read_int32 (struct GNUNET_BIO_ReadHandle *h, + const char *what, + int32_t *i) { - char what[PATH_MAX + 1024]; + int32_t big; - GNUNET_snprintf (what, sizeof(what), "%s:%d", file, line); - return GNUNET_BIO_read (h, what, result, len); + if (GNUNET_OK != GNUNET_BIO_read (h, what, &big, sizeof (int32_t))) + return GNUNET_SYSERR; + *i = ntohl (big); + return GNUNET_OK; +} + + +/** + * Read an (u)int64_t. + * + * @param h the IO handle to read from + * @param what describes what is being read (for error message creation) + * @param i where to store the data + * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + */ +int +GNUNET_BIO_read_int64 (struct GNUNET_BIO_ReadHandle *h, + const char *what, + int64_t *i) +{ + int64_t big; + if (GNUNET_OK != GNUNET_BIO_read (h, what, &big, sizeof (int64_t))) + return GNUNET_SYSERR; + *i = GNUNET_ntohll (big); + return GNUNET_OK; } /** - * Read 0-terminated string from a file. + * Read a 0-terminated string. * - * @param h handle to an open file + * @param h the IO handle to read from * @param what describes what is being read (for error message creation) - * @param result the buffer to store a pointer to the (allocated) string to - * (note that *result could be set to NULL as well) + * @param result the buffer to store a pointer to the (allocated) string + * can be NULL in certain cases * @param max_length maximum allowed length for the string - * @return #GNUNET_OK on success, #GNUNET_SYSERR on failure + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int GNUNET_BIO_read_string (struct GNUNET_BIO_ReadHandle *h, @@ -238,10 +384,21 @@ GNUNET_BIO_read_string (struct GNUNET_BIO_ReadHandle *h, char *buf; uint32_t big; - if (GNUNET_OK != GNUNET_BIO_read_int32 (h, &big)) + if (GNUNET_OK != GNUNET_BIO_read_int32 (h, + _ ("string length"), + (int32_t *) &big)) { - GNUNET_free_non_null (h->emsg); - GNUNET_asprintf (&h->emsg, _ ("Error reading length of string `%s'"), what); + char *tmp = h->emsg; + if (NULL != tmp) + GNUNET_asprintf (&h->emsg, + _ ("%s (while reading string `%s')"), + tmp, + what); + else + GNUNET_asprintf (&h->emsg, + _ ("Error reading length of string `%s'"), + what); + GNUNET_free_non_null (tmp); return GNUNET_SYSERR; } if (0 == big) @@ -263,7 +420,7 @@ GNUNET_BIO_read_string (struct GNUNET_BIO_ReadHandle *h, buf[--big] = '\0'; if (0 == big) return GNUNET_OK; - if (GNUNET_OK != GNUNET_BIO_read (h, what, buf, big)) + if (GNUNET_OK != GNUNET_BIO_read (h, what, buf, (int32_t) big)) { GNUNET_free (buf); *result = NULL; @@ -274,11 +431,11 @@ GNUNET_BIO_read_string (struct GNUNET_BIO_ReadHandle *h, /** - * Read metadata container from a file. + * Read a metadata container from a handle. * - * @param h handle to an open file + * @param h the IO handle to read from * @param what describes what is being read (for error message creation) - * @param result the buffer to store a pointer to the (allocated) metadata + * @param result the buffer to store a pointer to the (allocated) metadata. * @return #GNUNET_OK on success, #GNUNET_SYSERR on failure */ int @@ -290,106 +447,70 @@ GNUNET_BIO_read_meta_data (struct GNUNET_BIO_ReadHandle *h, char *buf; struct GNUNET_CONTAINER_MetaData *meta; - if (GNUNET_OK != GNUNET_BIO_read_int32 (h, (int32_t *) &size)) + if (GNUNET_OK != GNUNET_BIO_read_int32 (h, + _ ("metadata length"), + (int32_t *) &size)) return GNUNET_SYSERR; - if (size == 0) + if (0 == size) { *result = NULL; return GNUNET_OK; } - if (size > MAX_META_DATA) + if (MAX_META_DATA < size) { - GNUNET_asprintf (&h->emsg, - _ ("Serialized metadata `%s' larger than allowed (%u>%u)"), - what, - size, - MAX_META_DATA); + GNUNET_asprintf ( + &h->emsg, + _ ("Serialized metadata `%s' larger than allowed (%u > %u)"), + what, + size, + MAX_META_DATA); return GNUNET_SYSERR; } buf = GNUNET_malloc (size); - if (GNUNET_OK != GNUNET_BIO_read (h, what, buf, size)) + if (GNUNET_OK != GNUNET_BIO_read (h, what, buf, (int32_t) size)) { GNUNET_free (buf); return GNUNET_SYSERR; } meta = GNUNET_CONTAINER_meta_data_deserialize (buf, size); + GNUNET_free (buf); if (NULL == meta) { - GNUNET_free (buf); - GNUNET_asprintf (&h->emsg, _ ("Metadata `%s' failed to deserialize"), what); + GNUNET_asprintf (&h->emsg, _ ("Failed to deserialize metadata `%s'"), what); return GNUNET_SYSERR; } - GNUNET_free (buf); *result = meta; return GNUNET_OK; } -/** - * Read an (u)int32_t. - * - * @param h hande to open file - * @param file name of the source file - * @param line line number in the source file - * @param i address of 32-bit integer to read - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error - */ -int -GNUNET_BIO_read_int32__ (struct GNUNET_BIO_ReadHandle *h, - const char *file, - int line, - int32_t *i) -{ - int32_t big; - - if (GNUNET_OK != GNUNET_BIO_read_fn (h, file, line, &big, sizeof(int32_t))) - return GNUNET_SYSERR; - *i = ntohl (big); - return GNUNET_OK; -} - - -/** - * Read an (u)int64_t. - * - * @param h hande to open file - * @param file name of the source file - * @param line line number in the source file - * @param i address of 64-bit integer to read - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error - */ -int -GNUNET_BIO_read_int64__ (struct GNUNET_BIO_ReadHandle *h, - const char *file, - int line, - int64_t *i) -{ - int64_t big; - - if (GNUNET_OK != GNUNET_BIO_read_fn (h, file, line, &big, sizeof(int64_t))) - return GNUNET_SYSERR; - *i = GNUNET_ntohll (big); - return GNUNET_OK; -} - - /** * Handle for buffered writing. */ struct GNUNET_BIO_WriteHandle { /** - * Underlying file handle. + * The "backend" type. + */ + enum IOType type; + + /** + * Handle to a file on disk, if @e type is #IO_FILE. */ struct GNUNET_DISK_FileHandle *fd; /** - * I/O buffer. Do not free, allocated at the end of the struct. + * Error message. NULL is there were no errors. + */ + char *emsg; + + /** + * I/O buffer. Do @b not free! */ char *buffer; /** - * Number of bytes already in @e buffer. + * Number of bytes available in the @e buffer. */ size_t have; @@ -403,7 +524,7 @@ struct GNUNET_BIO_WriteHandle /** * Open a file for writing. * - * @param fn file name to be opened + * @param fn name of file to open * @return IO handle on success, NULL on error */ struct GNUNET_BIO_WriteHandle * @@ -414,7 +535,8 @@ GNUNET_BIO_write_open (const char *fn) fd = GNUNET_DISK_file_open (fn, - GNUNET_DISK_OPEN_WRITE | GNUNET_DISK_OPEN_TRUNCATE + GNUNET_DISK_OPEN_WRITE + | GNUNET_DISK_OPEN_TRUNCATE | GNUNET_DISK_OPEN_CREATE, GNUNET_DISK_PERM_USER_READ | GNUNET_DISK_PERM_USER_WRITE); @@ -429,26 +551,69 @@ GNUNET_BIO_write_open (const char *fn) /** - * Close an open file for writing. + * Create a handle from an existing buffer. + * + * @param buffer the buffer to use + * @param size the size in bytes of the buffer + (can be smaller than the real allocated size) + * @return IO handle on success, NULL on error + */ +struct GNUNET_BIO_WriteHandle * +GNUNET_BIO_write_create (void *buffer, size_t size) +{ + struct GNUNET_BIO_WriteHandle *h; + if (NULL == buffer || 0 == size) + return NULL; + h = GNUNET_new (struct GNUNET_BIO_WriteHandle); + h->buffer = buffer; + h->size = size; + h->type = IO_BUFFER; + return h; +} + + +/** + * Destroy an IO handle. If the handle was using a file, the file will be + * closed. If the handle was using a buffer allocated externally, the buffer + * will @b not be freed. * - * @param h file handle + * @param h the handle + * @param emsg set to the (allocated) error message + * if emsg is present, the return value is #GNUNET_SYSERR * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_write_close (struct GNUNET_BIO_WriteHandle *h) +GNUNET_BIO_write_destroy (struct GNUNET_BIO_WriteHandle *h, char **emsg) { - int ret; + int err; - ret = GNUNET_SYSERR; - if ((NULL != h->fd) && (GNUNET_OK == (ret = GNUNET_BIO_flush (h)))) - GNUNET_DISK_file_close (h->fd); + err = (NULL == h->emsg) ? GNUNET_OK : GNUNET_SYSERR; + if (NULL != emsg) + *emsg = h->emsg; + else + GNUNET_free_non_null (h->emsg); + switch (h->type) + { + case IO_FILE: + if (GNUNET_OK != GNUNET_BIO_flush (h)) + err = GNUNET_SYSERR; + else + GNUNET_DISK_file_close (h->fd); + break; + case IO_BUFFER: + break; + default: + break; + } GNUNET_free (h); - return ret; + return err; } /** - * Force a buffered writer to flush its buffer + * Force a file-based buffered writer to flush its buffer. + * If the handle does not use a file, this function simply returns #GNUNET_OK + * without doing anything. * * @param h the writer handle * @return #GNUNET_OK upon success. Upon failure #GNUNET_SYSERR is returned and @@ -459,12 +624,17 @@ GNUNET_BIO_flush (struct GNUNET_BIO_WriteHandle *h) { ssize_t ret; + if (IO_FILE != h->type) + return GNUNET_OK; + ret = GNUNET_DISK_file_write (h->fd, h->buffer, h->have); if (ret != (ssize_t) h->have) { GNUNET_DISK_file_close (h->fd); h->fd = NULL; - return GNUNET_SYSERR; /* error */ + GNUNET_free_non_null (h->emsg); + GNUNET_asprintf (&h->emsg, _ ("Flushing buffer")); + return GNUNET_SYSERR; } h->have = 0; return GNUNET_OK; @@ -472,96 +642,221 @@ GNUNET_BIO_flush (struct GNUNET_BIO_WriteHandle *h) /** - * Write a buffer to a file. + * Internal function used to write to a file. + * The written data is first buffered in memory so that disk activity is + * generated only when necessary. * - * @param h handle to open file + * @param h the IO handle + * @param what describes what is being written (for error message creation) * @param buffer the data to write - * @param n number of bytes to write - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @param n how many bytes should be written + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_write (struct GNUNET_BIO_WriteHandle *h, - const void *buffer, - size_t n) +write_file (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const char *buffer, + size_t n) { - const char *src = buffer; size_t min; size_t pos; - if (NULL == h->fd) - return GNUNET_SYSERR; pos = 0; do { - /* first, just use buffer */ + /* Try to use the buffer first */ min = h->size - h->have; - if (min > n - pos) + if (n - pos < min) min = n - pos; - GNUNET_memcpy (&h->buffer[h->have], &src[pos], min); + GNUNET_memcpy (&h->buffer[h->have], &buffer[pos], min); pos += min; h->have += min; - if (pos == n) - return GNUNET_OK; /* done */ + if (n == pos) + return GNUNET_OK; GNUNET_assert (h->have == h->size); if (GNUNET_OK != GNUNET_BIO_flush (h)) - return GNUNET_SYSERR; /* error */ + return GNUNET_SYSERR; } - while (pos < n); /* should always be true */ + while (pos < n); GNUNET_break (0); return GNUNET_OK; } /** - * Write a string to a file. + * Internal function to write to a buffer. + * The data is simply copied from the argument buffer to the destination + * buffer. Additionally, attempting to write too much data will result in an + * error instead of implicitly writing less data. * - * @param h handle to open file - * @param s string to write (can be NULL) - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @param h the IO handle + * @param what describes what is being written (for error message creation) + * @param buffer the data to write + * @param n how many bytes should be written + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise + */ +int +write_buffer (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const char *buffer, + size_t n) +{ + if (0 == n) + return GNUNET_OK; + if (h->size - h->have < n) + { + GNUNET_asprintf (&h->emsg, + _ ("Error while writing `%s': %s"), + what, + _ ("Not enough space in buffer")); + return GNUNET_SYSERR; + } + GNUNET_memcpy (h->buffer+h->have, buffer, + n); + h->have += n; + return GNUNET_OK; +} + + +/** + * Write a buffer to an IO handle. + * The destination depends on how the handle was created. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param buffer the data to write + * @param n number of bytes to write + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise + */ +int +GNUNET_BIO_write (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const void *buffer, + size_t n) +{ + const char *src = buffer; + int ret = GNUNET_OK; + + if (NULL != h->emsg) + return GNUNET_SYSERR; + + switch (h->type) + { + case IO_FILE: + ret = write_file (h, what, src, n); + break; + case IO_BUFFER: + ret = write_buffer (h, what, src, n); + break; + default: + break; + } + + return ret; +} + + +/** + * Write a (u)int32 to an IO handle. + * + * @param h the IO handle + * @param what what is being written (for error message creation) + * @param i 32-big integer to write + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise + */ +int +GNUNET_BIO_write_int32 (struct GNUNET_BIO_WriteHandle *h, + const char *what, + int32_t i) +{ + int32_t big; + + big = htonl (i); + return GNUNET_BIO_write (h, what, &big, sizeof (int32_t)); +} + + +/** + * Write a (u)int64 to an IO handle. + * + * @param h the IO handle + * @param what what is being written (for error message creation) + * @param i 64-bit integer to write + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise + */ +int +GNUNET_BIO_write_int64 (struct GNUNET_BIO_WriteHandle *h, + const char *what, + int64_t i) +{ + int64_t big; + + big = GNUNET_htonll (i); + return GNUNET_BIO_write (h, what, &big, sizeof (int64_t)); +} + + +/** + * Write a string to an IO handle. + * + * @param h the IO handle + * @param what what is being written (for error message creation) + * @param s 0-terminated string to write (can be NULL) + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_write_string (struct GNUNET_BIO_WriteHandle *h, const char *s) +GNUNET_BIO_write_string (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const char *s) { - uint32_t slen; + uint32_t len; - slen = (uint32_t) ((s == NULL) ? 0 : strlen (s) + 1); - if (GNUNET_OK != GNUNET_BIO_write_int32 (h, slen)) + len = (uint32_t) ((s == NULL) ? 0 : strlen (s) + 1); + if (GNUNET_OK != GNUNET_BIO_write_int32 (h, _ ("string length"), len)) return GNUNET_SYSERR; - if (0 != slen) - return GNUNET_BIO_write (h, s, slen - 1); + if (0 != len) + return GNUNET_BIO_write (h, what, s, len - 1); return GNUNET_OK; } /** - * Write metadata container to a file. + * Write metadata container to a IO handle. * - * @param h handle to open file + * @param h the IO handle + * @param what what is being written (for error message creation) * @param m metadata to write - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int GNUNET_BIO_write_meta_data (struct GNUNET_BIO_WriteHandle *h, + const char *what, const struct GNUNET_CONTAINER_MetaData *m) { ssize_t size; char *buf; - if (m == NULL) - return GNUNET_BIO_write_int32 (h, 0); + if (NULL == m) + return GNUNET_BIO_write_int32 (h, what, 0); buf = NULL; size = GNUNET_CONTAINER_meta_data_serialize ( m, &buf, MAX_META_DATA, GNUNET_CONTAINER_META_DATA_SERIALIZE_PART); - if (size == -1) + if (-1 == size) { GNUNET_free (buf); + GNUNET_free_non_null (h->emsg); + GNUNET_asprintf (&h->emsg, + _ ("Failed to serialize metadata `%s'"), + what); return GNUNET_SYSERR; } - if ((GNUNET_OK != GNUNET_BIO_write_int32 (h, (uint32_t) size)) || - (GNUNET_OK != GNUNET_BIO_write (h, buf, size))) + if ((GNUNET_OK != GNUNET_BIO_write_int32 (h, + _ ("metadata length"), + (uint32_t) size)) + || (GNUNET_OK != GNUNET_BIO_write (h, what, buf, size))) { GNUNET_free (buf); return GNUNET_SYSERR; @@ -572,36 +867,497 @@ GNUNET_BIO_write_meta_data (struct GNUNET_BIO_WriteHandle *h, /** - * Write an (u)int32_t. + * Create a read spec to read a certain amount of bytes. * - * @param h hande to open file - * @param i 32-bit integer to write - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @param size how many bytes should be read + * @return the read operation specification */ -int -GNUNET_BIO_write_int32 (struct GNUNET_BIO_WriteHandle *h, int32_t i) +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_object (struct GNUNET_BIO_ReadHandle *h, + const char *what, + void *result, + size_t size) { - int32_t big; + struct GNUNET_BIO_ReadSpec rs = { + .what = what, + .type = GNUNET_BIO_SPEC_OBJECT, + .h = h, + .size = size, + .result = result, + }; + + return rs; +} - big = htonl (i); - return GNUNET_BIO_write (h, &big, sizeof(int32_t)); + +/** + * Create a read spec to read an (u)int32. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_int32 (struct GNUNET_BIO_ReadHandle *h, + const char *what, + int32_t *result) +{ + struct GNUNET_BIO_ReadSpec rs = { + .what = what, + .type = GNUNET_BIO_SPEC_INT32, + .h = h, + .size = sizeof (int32_t), + .result = result, + }; + + return rs; } /** - * Write an (u)int64_t. + * Create a read spec to read an (u)int64. * - * @param h hande to open file - * @param i 64-bit integer to write - * @return #GNUNET_OK on success, #GNUNET_SYSERR on error + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_int64 (struct GNUNET_BIO_ReadHandle *h, + const char *what, + int64_t *result) +{ + struct GNUNET_BIO_ReadSpec rs = { + .what = what, + .type = GNUNET_BIO_SPEC_INT64, + .h = h, + .size = sizeof (int64_t), + .result = result, + }; + + return rs; +} + + +/** + * Create a read spec to read a float. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_float (struct GNUNET_BIO_ReadHandle *h, + const char *what, + float *result) +{ + struct GNUNET_BIO_ReadSpec rs = { + .what = what, + .type = GNUNET_BIO_SPEC_FLOAT, + .h = h, + .size = sizeof (float), + .result = result, + }; + + return rs; +} + + +/** + * Create a read spec to read a double. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store the data + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_double (struct GNUNET_BIO_ReadHandle *h, + const char *what, + double *result) +{ + struct GNUNET_BIO_ReadSpec rs = { + .what = what, + .type = GNUNET_BIO_SPEC_DOUBLE, + .h = h, + .size = sizeof (double), + .result = result, + }; + + return rs; +} + + +/** + * Create a read spec to read a 0-terminated string. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result where to store a pointer to the (allocated) string + * can be NULL in certain cases + * @param max_length maximum allowed length for the string + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_string (struct GNUNET_BIO_ReadHandle *h, + const char *what, + char **result, + size_t max_length) +{ + struct GNUNET_BIO_ReadSpec rs = { + .what = what, + .type = GNUNET_BIO_SPEC_STRING, + .h = h, + .size = max_length, + .result = result, + }; + + return rs; +} + + +/** + * Create a read spec to read a metadata container. + * + * @param h the IO handle to read from + * @param what what is being read (for error message creation) + * @param result the buffer to store a pointer to the (allocated) metadata. + * @return the read operation specification + */ +struct GNUNET_BIO_ReadSpec +GNUNET_BIO_read_spec_meta_data (struct GNUNET_BIO_ReadHandle *h, + const char *what, + struct GNUNET_CONTAINER_MetaData **result) +{ + struct GNUNET_BIO_ReadSpec rs = { + .what = what, + .type = GNUNET_BIO_SPEC_META, + .h = h, + .size = MAX_META_DATA, + .result = result, + }; + + return rs; +} + + +/** + * Execute the provided read operations. + * + * @param rs an array of read specs + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise */ int -GNUNET_BIO_write_int64 (struct GNUNET_BIO_WriteHandle *h, int64_t i) +GNUNET_BIO_read_spec_commit (struct GNUNET_BIO_ReadSpec *rs) { - int64_t big; + int ret; - big = GNUNET_htonll (i); - return GNUNET_BIO_write (h, &big, sizeof(int64_t)); + for (size_t i=0; NULL!=rs[i].h; ++i) + { + if (NULL != rs[i].h->emsg) + return GNUNET_SYSERR; + switch (rs[i].type) + { + case GNUNET_BIO_SPEC_OBJECT: + ret = GNUNET_BIO_read (rs[i].h, + rs[i].what, + (char *) (rs[i].result), + rs[i].size); + break; + case GNUNET_BIO_SPEC_INT32: + ret = GNUNET_BIO_read_int32 (rs[i].h, + rs[i].what, + (int32_t *) (rs[i].result)); + break; + case GNUNET_BIO_SPEC_INT64: + ret = GNUNET_BIO_read_int64 (rs[i].h, + rs[i].what, + (int64_t *) (rs[i].result)); + break; + case GNUNET_BIO_SPEC_FLOAT: + ret = GNUNET_BIO_read_float (rs[i].h, + rs[i].what, + (float *) (rs[i].result)); + break; + case GNUNET_BIO_SPEC_DOUBLE: + ret = GNUNET_BIO_read_double (rs[i].h, + rs[i].what, + (double *) (rs[i].result)); + break; + case GNUNET_BIO_SPEC_STRING: + ret = GNUNET_BIO_read_string (rs[i].h, + rs[i].what, + (char **) (rs[i].result), + rs[i].size); + break; + case GNUNET_BIO_SPEC_META: + ret = GNUNET_BIO_read_meta_data ( + rs[i].h, + rs[i].what, + (struct GNUNET_CONTAINER_MetaData **) (rs[i].result)); + break; + default: + GNUNET_free_non_null (rs[i].h->emsg); + GNUNET_asprintf (&(rs[i].h->emsg), + _ ("Invalid data type while reading `%s'"), + rs[i].what); + ret = GNUNET_SYSERR; + break; + } + if (GNUNET_OK != ret) + return GNUNET_SYSERR; + } + return GNUNET_OK; +} + + +/** + * Create a write spec to write a certain amount of bytes. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source the data to write to the handle + * @param size how many bytes should be written + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_object (struct GNUNET_BIO_WriteHandle *h, + const char *what, + void *source, + size_t size) +{ + struct GNUNET_BIO_WriteSpec ws = { + .what = what, + .type = GNUNET_BIO_SPEC_OBJECT, + .h = h, + .size = size, + .source = source, + }; + + return ws; +} + + +/** + * Create a write spec to write an (u)int32. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source a pointer to a 32-bit integer + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_int32 (struct GNUNET_BIO_WriteHandle *h, + const char *what, + int32_t *source) +{ + struct GNUNET_BIO_WriteSpec ws = { + .what = what, + .type = GNUNET_BIO_SPEC_INT32, + .h = h, + .size = sizeof (int32_t), + .source = source, + }; + + return ws; +} + + +/** + * Create a write spec to write an (u)int64. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source a pointer to a 64-bit integer + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_int64 (struct GNUNET_BIO_WriteHandle *h, + const char *what, + int64_t *source) +{ + struct GNUNET_BIO_WriteSpec ws = { + .what = what, + .type = GNUNET_BIO_SPEC_INT64, + .h = h, + .size = sizeof (int64_t), + .source = source, + }; + + return ws; +} + + +/** + * Create a write spec to write a float. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source a pointer to a float + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_float (struct GNUNET_BIO_WriteHandle *h, + const char *what, + float *source) +{ + struct GNUNET_BIO_WriteSpec ws = { + .what = what, + .type = GNUNET_BIO_SPEC_FLOAT, + .h = h, + .size = sizeof (float), + .source = source, + }; + + return ws; +} + + +/** + * Create a write spec to write a double. + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source a pointer to a double + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_double (struct GNUNET_BIO_WriteHandle *h, + const char *what, + double *source) +{ + struct GNUNET_BIO_WriteSpec ws = { + .what = what, + .type = GNUNET_BIO_SPEC_DOUBLE, + .h = h, + .size = sizeof (double), + .source = source, + }; + + return ws; +} + + +/** + * Create a write spec to write a string + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source the 0-terminated string to write + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_string (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const char *source) +{ + struct GNUNET_BIO_WriteSpec ws = { + .what = what, + .type = GNUNET_BIO_SPEC_STRING, + .h = h, + .size = ((NULL == source) ? 0 : strlen (source)), + .source = source, + }; + + return ws; +} + + +/** + * Create a write spec to write a metadata container + * + * @param h the IO handle to write to + * @param what what is being written (for error message creation) + * @param source the metadata to write + * @return the write operation specification + */ +struct GNUNET_BIO_WriteSpec +GNUNET_BIO_write_spec_meta_data (struct GNUNET_BIO_WriteHandle *h, + const char *what, + const struct GNUNET_CONTAINER_MetaData *source) +{ + struct GNUNET_BIO_WriteSpec ws = { + .what = what, + .type = GNUNET_BIO_SPEC_META, + .h = h, + .size = MAX_META_DATA, + .source = source, + }; + + return ws; +} + + +/** + * Execute the provided write operations. + * + * @param ws an array of write specs + * @return #GNUNET_OK on success, #GNUNET_SYSERR otherwise + */ +int +GNUNET_BIO_write_spec_commit (struct GNUNET_BIO_WriteSpec *ws) +{ + int ret; + for (size_t i=0; NULL!=ws[i].h; ++i) + { + if (NULL != ws[i].h->emsg) + return GNUNET_SYSERR; + switch (ws[i].type) + { + case GNUNET_BIO_SPEC_OBJECT: + ret = GNUNET_BIO_write (ws[i].h, + ws[i].what, + ws[i].source, + ws[i].size); + break; + case GNUNET_BIO_SPEC_INT32: + ret = GNUNET_BIO_write_int32 (ws[i].h, + ws[i].what, + *((int32_t *) ws[i].source)); + break; + case GNUNET_BIO_SPEC_INT64: + ret = GNUNET_BIO_write_int64 (ws[i].h, + ws[i].what, + *((int64_t *) ws[i].source)); + break; + case GNUNET_BIO_SPEC_FLOAT: + ret = GNUNET_BIO_write_float (ws[i].h, + ws[i].what, + *((float *) ws[i].source)); + break; + case GNUNET_BIO_SPEC_DOUBLE: + ret = GNUNET_BIO_write_double (ws[i].h, + ws[i].what, + *((double *) ws[i].source)); + break; + case GNUNET_BIO_SPEC_STRING: + ret = GNUNET_BIO_write_string (ws[i].h, + ws[i].what, + (const char *) ws[i].source); + break; + case GNUNET_BIO_SPEC_META: + ret = GNUNET_BIO_write_meta_data ( + ws[i].h, + ws[i].what, + (const struct GNUNET_CONTAINER_MetaData *) ws[i].source); + break; + default: + GNUNET_free_non_null (ws[i].h->emsg); + GNUNET_asprintf (&(ws[i].h->emsg), + _ ("Invalid data type while writing `%s'"), + ws[i].what); + ret = GNUNET_SYSERR; + break; + } + if (GNUNET_OK != ret) + return GNUNET_SYSERR; + } + return GNUNET_OK; } -- 2.26.2
